Sep 15 06:36:30.462258 ip-10-0-128-50 systemd[1]: Starting Kubernetes Kubelet... Sep 15 06:36:30.872185 ip-10-0-128-50 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 06:36:30.872185 ip-10-0-128-50 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 15 06:36:30.872185 ip-10-0-128-50 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 06:36:30.872185 ip-10-0-128-50 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 15 06:36:30.872185 ip-10-0-128-50 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 06:36:30.874528 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.874418 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 15 06:36:30.878129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878114 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:30.878129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878129 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878133 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878136 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878139 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878143 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878146 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878149 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878152 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878154 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878157 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878160 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878163 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878180 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878183 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878186 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878188 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878191 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878195 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878200 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:30.878193 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878203 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878206 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878209 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878211 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878214 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878217 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878219 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878222 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878224 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878227 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878231 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878234 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878237 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878240 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878243 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878246 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878248 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878251 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878253 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:30.878650 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878256 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878259 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878261 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878263 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878266 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878268 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878271 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878274 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878276 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878279 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878281 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878284 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878286 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878289 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878291 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878294 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878795 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878799 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878802 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878805 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:30.879249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878810 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878813 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878816 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878819 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878822 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878825 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878827 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878832 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878835 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878839 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878842 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878847 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878850 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878853 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878855 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878858 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878860 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878863 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878866 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:30.880065 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878869 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878872 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878874 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878877 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878880 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878882 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878885 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878888 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878891 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878893 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878896 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878898 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878901 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878903 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878906 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878908 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878911 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878914 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878916 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878918 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:30.880702 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.878921 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881050 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881062 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881065 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881069 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881072 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881076 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881079 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881081 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881084 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881087 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881090 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881093 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881095 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881098 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881101 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881103 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881106 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881108 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:30.881213 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881111 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881116 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881119 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881121 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881124 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881126 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881130 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881134 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881137 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881140 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881142 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881145 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881147 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881150 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881152 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881155 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881158 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881161 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881163 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881166 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:30.881684 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881169 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881171 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881174 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881176 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881179 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881181 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881184 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881186 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881188 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881191 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881193 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881196 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881198 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881201 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881203 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881206 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881208 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881210 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881213 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881215 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:30.882249 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881218 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881221 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881223 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881226 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881229 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881231 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881234 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881238 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881240 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881243 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881245 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881248 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881251 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881253 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881255 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881258 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881260 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881263 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881265 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881267 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:30.882734 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881271 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881273 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881276 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881278 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881281 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881289 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881292 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881295 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881297 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881301 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881303 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881306 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881308 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881312 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881316 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881319 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881323 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881327 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881330 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:30.883252 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881333 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881337 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881715 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881720 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881723 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881725 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881728 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881731 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881734 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881736 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881739 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881741 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881743 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881746 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881748 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881751 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881753 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881756 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881759 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881762 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:30.883723 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881764 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881767 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881769 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881772 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881774 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881776 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881779 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881781 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881784 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881786 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881788 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881791 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881794 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881797 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881800 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881802 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881805 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881807 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881811 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:30.884229 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881815 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881817 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881820 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881825 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881828 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881831 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881834 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881836 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881839 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881841 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881843 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881846 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881849 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881852 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881854 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881857 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881860 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881862 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881864 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:30.884682 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881867 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881869 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881872 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881874 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881877 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881879 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881882 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881885 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881888 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881890 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881893 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881896 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881899 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881902 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881904 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881907 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881910 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881912 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881915 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881917 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:30.885160 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881919 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881922 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881924 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881926 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881929 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881931 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881934 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881937 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881939 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881941 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881944 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881946 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881948 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881951 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881953 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881956 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881958 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881961 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881964 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881984 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:30.885641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881989 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881992 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.881995 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882071 2581 flags.go:64] FLAG: --address="0.0.0.0" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882104 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882110 2581 flags.go:64] FLAG: --anonymous-auth="true" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882115 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882119 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882122 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882130 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882134 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882137 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882140 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882144 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882147 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882150 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882153 2581 flags.go:64] FLAG: --cgroup-root="" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882156 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882159 2581 flags.go:64] FLAG: --client-ca-file="" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882162 2581 flags.go:64] FLAG: --cloud-provider="external" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882165 2581 flags.go:64] FLAG: --cluster-dns="[]" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882170 2581 flags.go:64] FLAG: --cluster-domain="" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882173 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 15 06:36:30.886140 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882176 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882179 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882182 2581 flags.go:64] FLAG: --container-log-max-files="5" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882186 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882189 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882192 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882195 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882198 2581 flags.go:64] FLAG: --contention-profiling="false" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882201 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882204 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882207 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882210 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882213 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882216 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882219 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882222 2581 flags.go:64] FLAG: --enable-load-reader="false" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882224 2581 flags.go:64] FLAG: --enable-server="true" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882227 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882231 2581 flags.go:64] FLAG: --event-burst="100" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882234 2581 flags.go:64] FLAG: --event-qps="50" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882237 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882240 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882243 2581 flags.go:64] FLAG: --eviction-hard="" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882246 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 15 06:36:30.886692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882249 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882252 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882255 2581 flags.go:64] FLAG: --eviction-soft="" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882258 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882261 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882264 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882267 2581 flags.go:64] FLAG: --experimental-mounter-path="" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882270 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882273 2581 flags.go:64] FLAG: --fail-swap-on="true" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882275 2581 flags.go:64] FLAG: --feature-gates="" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882279 2581 flags.go:64] FLAG: --file-check-frequency="20s" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882282 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882285 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882288 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882291 2581 flags.go:64] FLAG: --healthz-port="10248" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882294 2581 flags.go:64] FLAG: --help="false" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882297 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-128-50.ec2.internal" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882300 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882302 2581 flags.go:64] FLAG: --http-check-frequency="20s" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882305 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882309 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882312 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882314 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 15 06:36:30.887275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882317 2581 flags.go:64] FLAG: --image-service-endpoint="" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882320 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882322 2581 flags.go:64] FLAG: --kube-api-burst="100" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882325 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882328 2581 flags.go:64] FLAG: --kube-api-qps="50" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882331 2581 flags.go:64] FLAG: --kube-reserved="" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882333 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882336 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882339 2581 flags.go:64] FLAG: --kubelet-cgroups="" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882344 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882347 2581 flags.go:64] FLAG: --lock-file="" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882349 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882352 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882355 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882360 2581 flags.go:64] FLAG: --log-json-split-stream="false" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882364 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882368 2581 flags.go:64] FLAG: --log-text-split-stream="false" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882370 2581 flags.go:64] FLAG: --logging-format="text" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882373 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882376 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882379 2581 flags.go:64] FLAG: --manifest-url="" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882382 2581 flags.go:64] FLAG: --manifest-url-header="" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882386 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882389 2581 flags.go:64] FLAG: --max-open-files="1000000" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882393 2581 flags.go:64] FLAG: --max-pods="110" Sep 15 06:36:30.887915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882396 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882399 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882401 2581 flags.go:64] FLAG: --memory-manager-policy="None" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882404 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882407 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882410 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882413 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882421 2581 flags.go:64] FLAG: --node-status-max-images="50" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882423 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882426 2581 flags.go:64] FLAG: --oom-score-adj="-999" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882430 2581 flags.go:64] FLAG: --pod-cidr="" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882432 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882437 2581 flags.go:64] FLAG: --pod-manifest-path="" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882440 2581 flags.go:64] FLAG: --pod-max-pids="-1" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882443 2581 flags.go:64] FLAG: --pods-per-core="0" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882446 2581 flags.go:64] FLAG: --port="10250" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882450 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882453 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e5c64bc000d319ff" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882456 2581 flags.go:64] FLAG: --qos-reserved="" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882459 2581 flags.go:64] FLAG: --read-only-port="10255" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882462 2581 flags.go:64] FLAG: --register-node="true" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882465 2581 flags.go:64] FLAG: --register-with-taints="" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882468 2581 flags.go:64] FLAG: --registry-burst="10" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882471 2581 flags.go:64] FLAG: --registry-qps="5" Sep 15 06:36:30.888670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882477 2581 flags.go:64] FLAG: --reserved-cpus="" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882480 2581 flags.go:64] FLAG: --reserved-memory="" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882483 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882486 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882489 2581 flags.go:64] FLAG: --rotate-certificates="false" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882492 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882494 2581 flags.go:64] FLAG: --runonce="false" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882497 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882500 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882503 2581 flags.go:64] FLAG: --seccomp-default="false" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882505 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882508 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882511 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882514 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882517 2581 flags.go:64] FLAG: --storage-driver-password="root" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882519 2581 flags.go:64] FLAG: --storage-driver-secure="false" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882522 2581 flags.go:64] FLAG: --storage-driver-table="stats" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882525 2581 flags.go:64] FLAG: --storage-driver-user="root" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882528 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882531 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882534 2581 flags.go:64] FLAG: --system-cgroups="" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882536 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882541 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882544 2581 flags.go:64] FLAG: --tls-cert-file="" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882548 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 15 06:36:30.889292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882553 2581 flags.go:64] FLAG: --tls-min-version="" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882555 2581 flags.go:64] FLAG: --tls-private-key-file="" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882558 2581 flags.go:64] FLAG: --topology-manager-policy="none" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882561 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882564 2581 flags.go:64] FLAG: --topology-manager-scope="container" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882566 2581 flags.go:64] FLAG: --v="2" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882570 2581 flags.go:64] FLAG: --version="false" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882577 2581 flags.go:64] FLAG: --vmodule="" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882581 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.882584 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882694 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882698 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882701 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882704 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882707 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882709 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882712 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882714 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882717 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882720 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882722 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882725 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:30.889910 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882727 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882730 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882732 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882735 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882737 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882740 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882744 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882748 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882750 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882754 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882757 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882760 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882762 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882765 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882767 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882770 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882772 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882776 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882779 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882781 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:30.890462 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882784 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882787 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882789 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882792 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882794 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882796 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882799 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882802 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882804 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882808 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882811 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882814 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882817 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882820 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882822 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882825 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882827 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882830 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882833 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:30.890963 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882835 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882838 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882842 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882845 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882848 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882850 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882853 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882855 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882858 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882861 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882866 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882869 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882872 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882875 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882877 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882880 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882883 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882886 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882888 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882891 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:30.891641 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882893 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882896 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882899 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882901 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882904 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882906 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882909 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882912 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882914 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882917 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882919 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882921 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882924 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882927 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882930 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882933 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882935 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882938 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882940 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882943 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:30.892198 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882945 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882948 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882952 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882954 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882957 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882959 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882962 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.882965 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.883709 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.892183 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 15 06:36:30.892695 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.892311 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 15 06:36:30.893164 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.893153 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 15 06:36:30.893201 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.893165 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 15 06:36:30.893232 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893225 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:30.893232 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893230 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:30.893232 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893233 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893237 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893240 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893243 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893246 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893248 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893252 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893254 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893257 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893260 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893262 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893266 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893268 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893271 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893274 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893276 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893279 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893281 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893284 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893286 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:30.893313 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893289 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893291 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893296 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893300 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893302 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893305 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893307 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893310 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893313 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893316 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893319 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893322 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893325 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893327 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893330 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893333 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893335 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893338 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893340 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:30.893804 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893343 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893345 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893348 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893350 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893353 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893356 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893358 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893360 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893363 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893365 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893368 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893370 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893373 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893375 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893378 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893381 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893384 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893386 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893389 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893391 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:30.894354 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893394 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893397 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893400 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893402 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893405 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893407 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893409 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893412 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893414 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893417 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893419 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893421 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893424 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893426 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893429 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893431 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893433 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893436 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893438 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893441 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:30.894843 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893443 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893446 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893449 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893451 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893455 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893458 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893462 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893466 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893470 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893473 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893475 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893479 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893482 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893484 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893487 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893490 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893493 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:30.895360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893495 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.893500 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893614 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893618 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893621 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893623 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893627 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893629 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893632 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893634 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893637 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893639 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893642 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893644 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893647 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:30.895772 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893650 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893652 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893655 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893657 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893660 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893662 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893664 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893668 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893670 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893673 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893675 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893678 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893680 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893682 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893685 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893688 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893690 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893693 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893695 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893698 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:30.896166 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893700 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893702 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893705 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893707 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893709 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893712 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893715 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893717 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893719 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893721 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893724 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893726 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893730 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893733 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893736 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893738 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893742 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893744 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893747 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:30.896656 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893750 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893753 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893757 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893760 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893763 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893765 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893768 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893770 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893773 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893776 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893778 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893780 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893783 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893785 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893787 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893790 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893792 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893795 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893797 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:30.897129 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893799 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893802 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893804 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893807 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893809 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893812 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893815 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893818 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893820 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893823 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893825 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893828 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893830 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893833 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893835 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893841 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893843 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893846 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893848 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893851 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:30.897584 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893853 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893856 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893858 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893861 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893864 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893866 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893870 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:30.893873 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.893877 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.894002 2581 server.go:962] "Client rotation is on, will bootstrap in background" Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.896019 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.897004 2581 server.go:1019] "Starting client certificate rotation" Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.897107 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 06:36:30.898191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.897142 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 06:36:30.899433 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.899420 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 06:36:30.900575 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.900557 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 06:36:30.909460 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.909442 2581 log.go:25] "Validated CRI v1 runtime API" Sep 15 06:36:30.912496 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.912476 2581 log.go:25] "Validated CRI v1 image API" Sep 15 06:36:30.913411 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.913396 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 15 06:36:30.916195 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.916174 2581 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d0c0aa23-aad9-4346-9c4c-0ac2e99114a6:/dev/nvme0n1p4 e0265e27-4824-4921-9dfd-4a8368e0e9b0:/dev/nvme0n1p3] Sep 15 06:36:30.916244 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.916197 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 15 06:36:30.920926 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.920910 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 06:36:30.922860 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.922748 2581 manager.go:216] Machine: {Timestamp:2026-09-15 06:36:30.921456925 +0000 UTC m=+0.344007382 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099208 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27b83b1c9650c450774e1d16d1e4c8 SystemUUID:ec27b83b-1c96-50c4-5077-4e1d16d1e4c8 BootID:559246cb-8c5f-4769-8c07-174de3530bef Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:da:a5:2f:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:da:a5:2f:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:d8:c7:70:1c:2f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 15 06:36:30.922860 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.922859 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 15 06:36:30.923004 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.922991 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 15 06:36:30.926353 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.926329 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 15 06:36:30.926488 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.926355 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-50.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 15 06:36:30.927031 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.927021 2581 topology_manager.go:138] "Creating topology manager with none policy" Sep 15 06:36:30.927066 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.927034 2581 container_manager_linux.go:309] "Creating device plugin manager" Sep 15 06:36:30.927066 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.927041 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 06:36:30.927692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.927683 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 06:36:30.928526 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.928516 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 15 06:36:30.932131 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932119 2581 state_mem.go:36] "Initialized new in-memory state store" Sep 15 06:36:30.932259 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932249 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 15 06:36:30.932301 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932282 2581 kubelet.go:486] "Attempting to sync node with API server" Sep 15 06:36:30.932301 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932291 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 15 06:36:30.932364 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932304 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 15 06:36:30.932364 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932313 2581 kubelet.go:398] "Adding apiserver pod source" Sep 15 06:36:30.932364 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932322 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 15 06:36:30.932696 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932684 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 06:36:30.932740 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.932703 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 06:36:30.934886 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.934869 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 15 06:36:30.942161 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.942144 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 15 06:36:30.942229 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.942176 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 15 06:36:30.944533 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944513 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 15 06:36:30.944533 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944534 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944535 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q477h" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944540 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944573 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944589 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944596 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944610 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944616 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944623 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944628 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944637 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 15 06:36:30.944643 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.944643 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 15 06:36:30.945440 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.945426 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 15 06:36:30.945440 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.945441 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 15 06:36:30.945532 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:30.945445 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 15 06:36:30.945532 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:30.945450 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-50.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 15 06:36:30.948945 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.948932 2581 server.go:1268] "Started kubelet" Sep 15 06:36:30.949044 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.949022 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 15 06:36:30.949178 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.949131 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 15 06:36:30.949251 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.949197 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 15 06:36:30.949821 ip-10-0-128-50 systemd[1]: Started Kubernetes Kubelet. Sep 15 06:36:30.950520 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.950501 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 15 06:36:30.952625 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.952603 2581 server.go:310] "Adding debug handlers to kubelet server" Sep 15 06:36:30.953660 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:30.953635 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 15 06:36:30.954095 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.954073 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q477h" Sep 15 06:36:30.956955 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.956937 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-50.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 15 06:36:30.957088 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.957071 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 15 06:36:30.957638 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.957618 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 15 06:36:30.957746 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:30.957725 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:30.957907 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:30.956935 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18d56b1b1157efc5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-09-15 06:36:30.948904901 +0000 UTC m=+0.371455358,LastTimestamp:2026-09-15 06:36:30.948904901 +0000 UTC m=+0.371455358,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Sep 15 06:36:30.958101 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.958085 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 15 06:36:30.958169 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.958087 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 15 06:36:30.958169 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.958133 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 15 06:36:30.958249 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.958228 2581 reconstruct.go:97] "Volume reconstruction finished" Sep 15 06:36:30.958249 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.958236 2581 reconciler.go:29] "Reconciler: start to sync state" Sep 15 06:36:30.958849 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.958829 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 15 06:36:30.958849 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.958852 2581 factory.go:55] Registering systemd factory Sep 15 06:36:30.958994 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.958863 2581 factory.go:223] Registration of the systemd container factory successfully Sep 15 06:36:30.959131 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.959113 2581 factory.go:153] Registering CRI-O factory Sep 15 06:36:30.959131 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.959129 2581 factory.go:223] Registration of the crio container factory successfully Sep 15 06:36:30.959267 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.959145 2581 factory.go:103] Registering Raw factory Sep 15 06:36:30.959267 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.959157 2581 manager.go:1196] Started watching for new ooms in manager Sep 15 06:36:30.959597 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.959581 2581 manager.go:349] Starting recovery of all containers Sep 15 06:36:30.962072 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.962048 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:30.965531 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:30.965506 2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-50.ec2.internal\" not found" node="ip-10-0-128-50.ec2.internal" Sep 15 06:36:30.969729 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.969712 2581 manager.go:354] Recovery completed Sep 15 06:36:30.973864 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.973852 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:30.975667 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.975649 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:30.975760 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.975686 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:30.975760 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.975701 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:30.976303 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.976291 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 15 06:36:30.976359 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.976303 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 15 06:36:30.976359 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.976318 2581 state_mem.go:36] "Initialized new in-memory state store" Sep 15 06:36:30.976448 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.976359 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:30.977088 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.977072 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:30.977162 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.977093 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:30.977162 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.977103 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:30.977698 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.977687 2581 policy_none.go:49] "None policy: Start" Sep 15 06:36:30.977742 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.977702 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 15 06:36:30.977742 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.977712 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 15 06:36:30.978791 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:30.978780 2581 policy_none.go:47] "Start" Sep 15 06:36:31.014581 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.014563 2581 manager.go:337] "Starting Device Plugin manager" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.014604 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.014617 2581 server.go:87] "Starting device plugin registration server" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.014870 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.014882 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.015067 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.015151 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.015160 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.015596 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 15 06:36:31.026856 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.015640 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.054776 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.054749 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 15 06:36:31.056113 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.056096 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 15 06:36:31.056200 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.056143 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 15 06:36:31.056200 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.056167 2581 kubelet.go:2448] "Starting kubelet main sync loop" Sep 15 06:36:31.056280 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.056202 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 15 06:36:31.058930 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.058913 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:31.115222 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.115196 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:31.122344 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.122259 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:31.122344 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.122289 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:31.122344 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.122299 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:31.122344 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.122324 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.130541 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.130526 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.130601 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.130548 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-50.ec2.internal\": node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.144227 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.144205 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.156282 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.156260 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal"] Sep 15 06:36:31.156360 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.156337 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:31.157747 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.157729 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:31.157810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.157760 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:31.157810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.157775 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:31.158935 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.158923 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:31.159078 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.159062 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.159139 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.159099 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:31.159639 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.159625 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:31.159703 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.159650 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:31.159703 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.159660 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:31.160569 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.160554 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:31.160642 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.160580 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:31.160642 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.160589 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:31.161605 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.161586 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.161711 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.161644 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:31.162357 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.162342 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:31.162425 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.162372 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:31.162425 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.162383 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:31.186603 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.186587 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-50.ec2.internal\" not found" node="ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.191107 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.191091 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-50.ec2.internal\" not found" node="ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.244322 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.244295 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.259568 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.259550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd7d5957c3c1631a2eca1f67dcc24ff3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"fd7d5957c3c1631a2eca1f67dcc24ff3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.259664 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.259575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd7d5957c3c1631a2eca1f67dcc24ff3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"fd7d5957c3c1631a2eca1f67dcc24ff3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.259664 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.259593 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cc9a0b5b683b9efc648017b7df9cf169-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"cc9a0b5b683b9efc648017b7df9cf169\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.344709 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.344674 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.360002 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.359961 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cc9a0b5b683b9efc648017b7df9cf169-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"cc9a0b5b683b9efc648017b7df9cf169\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.360105 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.360010 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd7d5957c3c1631a2eca1f67dcc24ff3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"fd7d5957c3c1631a2eca1f67dcc24ff3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.360105 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.360077 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd7d5957c3c1631a2eca1f67dcc24ff3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"fd7d5957c3c1631a2eca1f67dcc24ff3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.360105 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.360095 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cc9a0b5b683b9efc648017b7df9cf169-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"cc9a0b5b683b9efc648017b7df9cf169\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.360227 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.360119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd7d5957c3c1631a2eca1f67dcc24ff3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"fd7d5957c3c1631a2eca1f67dcc24ff3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.360227 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.360160 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd7d5957c3c1631a2eca1f67dcc24ff3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"fd7d5957c3c1631a2eca1f67dcc24ff3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.445481 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.445406 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.489888 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.489848 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.493785 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.493572 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Sep 15 06:36:31.546116 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.546086 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.646608 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.646576 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.747120 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.747051 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.847640 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.847618 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.897078 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.897054 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 15 06:36:31.897577 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.897204 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 15 06:36:31.897577 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.897244 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 15 06:36:31.948550 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:31.948524 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:31.959317 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.959287 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 06:31:32 +0000 UTC" deadline="2028-02-14 02:42:59.442161112 +0000 UTC" Sep 15 06:36:31.959317 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:31.959314 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12404h6m27.482850337s" Sep 15 06:36:32.049488 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:32.049396 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:32.101278 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:32.101092 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc9a0b5b683b9efc648017b7df9cf169.slice/crio-e98c3cfadd60d62f088fc3b65edf9fe22c762009787da6cad5447ea7d8705384 WatchSource:0}: Error finding container e98c3cfadd60d62f088fc3b65edf9fe22c762009787da6cad5447ea7d8705384: Status 404 returned error can't find the container with id e98c3cfadd60d62f088fc3b65edf9fe22c762009787da6cad5447ea7d8705384 Sep 15 06:36:32.104703 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:32.104687 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 06:36:32.149768 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:32.149740 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Sep 15 06:36:32.151584 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:32.151567 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:32.158449 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:32.158435 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Sep 15 06:36:32.169744 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:32.169728 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 06:36:32.171789 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:32.171777 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Sep 15 06:36:32.180229 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:32.180211 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 06:36:32.279918 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:32.279890 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:34.282384 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.282353 2581 apiserver.go:52] "Watching apiserver" Sep 15 06:36:34.290856 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.290824 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 15 06:36:34.292129 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.292098 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:34.293398 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.293378 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-hq4tx","openshift-multus/network-metrics-daemon-xm2jl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal","openshift-multus/multus-additional-cni-plugins-n287w","openshift-network-diagnostics/network-check-target-pwmjh","openshift-network-operator/iptables-alerter-p8pr4","openshift-ovn-kubernetes/ovnkube-node-6p2cg","kube-system/global-pull-secret-syncer-s4bpp","kube-system/konnectivity-agent-twhfn","kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal","openshift-cluster-node-tuning-operator/tuned-vvxb9","openshift-dns/node-resolver-mx7sh","openshift-image-registry/node-ca-tjm9m"] Sep 15 06:36:34.294844 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.294819 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:34.295890 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.295869 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.296986 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.296968 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.297448 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.297430 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9mtxn\"" Sep 15 06:36:34.297535 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.297466 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 15 06:36:34.297535 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.297494 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 15 06:36:34.298075 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.298058 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.298437 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.298418 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-26rps\"" Sep 15 06:36:34.298517 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.298483 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 15 06:36:34.298517 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.298509 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 15 06:36:34.298903 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.298883 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 15 06:36:34.299105 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.299089 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 15 06:36:34.299381 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.299365 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cch5p\"" Sep 15 06:36:34.299543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.299529 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 15 06:36:34.299619 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.299603 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.300038 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.300021 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 15 06:36:34.300157 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.300120 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jstt7\"" Sep 15 06:36:34.300341 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.300326 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 15 06:36:34.300415 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.300369 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 15 06:36:34.300528 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.300500 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:34.300594 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.300572 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:34.301457 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.301440 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 15 06:36:34.301559 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.301442 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 15 06:36:34.301602 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.301556 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 15 06:36:34.301799 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.301781 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zprl8\"" Sep 15 06:36:34.301871 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.301830 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 15 06:36:34.302218 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.302131 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 15 06:36:34.302218 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.302184 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.302314 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.302244 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:34.303520 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.303501 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.304960 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.304939 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:34.305064 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.305013 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:34.305725 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.305705 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 15 06:36:34.305822 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.305753 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 15 06:36:34.305822 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.305766 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 15 06:36:34.305822 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.305799 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c5z8w\"" Sep 15 06:36:34.305974 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.305882 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 15 06:36:34.307726 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.307700 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.310426 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.309770 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.310426 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.310330 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cmgxq\"" Sep 15 06:36:34.310426 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.310350 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 15 06:36:34.311342 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.311326 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.312741 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.312720 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 15 06:36:34.313486 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.313423 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 15 06:36:34.313576 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.313513 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8ln8x\"" Sep 15 06:36:34.314126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.314108 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 15 06:36:34.314283 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.314115 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 15 06:36:34.314345 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.314287 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 15 06:36:34.314525 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.314493 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 15 06:36:34.314525 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.314517 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tzjl6\"" Sep 15 06:36:34.314622 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.314530 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 15 06:36:34.314792 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.314773 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 15 06:36:34.315051 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315004 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-k8s-cni-cncf-io\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.315158 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315049 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-conf-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.315158 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c30094d0-3eb2-4349-b7c0-5f33d52d9e84-agent-certs\") pod \"konnectivity-agent-twhfn\" (UID: \"c30094d0-3eb2-4349-b7c0-5f33d52d9e84\") " pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:34.315158 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315093 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs6zb\" (UniqueName: \"kubernetes.io/projected/c148b74f-c481-4720-bf3d-50bd390af2da-kube-api-access-gs6zb\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.315158 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315113 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqfdr\" (UniqueName: \"kubernetes.io/projected/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-kube-api-access-fqfdr\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.315158 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315137 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.315359 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-system-cni-dir\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.315359 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315240 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-netns\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.315359 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315263 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-cni-bin\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.315359 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315291 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-hostroot\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.315359 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3eadb88e-afda-41c4-836a-c044d4a2628d-kubelet-config\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.315359 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315351 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.315685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z9cq\" (UniqueName: \"kubernetes.io/projected/9e225fca-64e1-4eb9-9424-c54b0b77cad6-kube-api-access-4z9cq\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:34.315685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315447 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-cni-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.315685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.315493 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-daemon-config\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.316855 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.316817 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c148b74f-c481-4720-bf3d-50bd390af2da-hosts-file\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.316941 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.316901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-socket-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.317000 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.316950 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-cni-binary-copy\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.317000 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.316981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3eadb88e-afda-41c4-836a-c044d4a2628d-dbus\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.317094 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.317094 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317059 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/363050d3-1590-4804-b6cc-48cecf994d16-serviceca\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.317209 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317115 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tkkp\" (UniqueName: \"kubernetes.io/projected/71955717-dffc-4ec7-8472-c320b24abff9-kube-api-access-7tkkp\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.317209 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317167 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c30094d0-3eb2-4349-b7c0-5f33d52d9e84-konnectivity-ca\") pod \"konnectivity-agent-twhfn\" (UID: \"c30094d0-3eb2-4349-b7c0-5f33d52d9e84\") " pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:34.317297 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317192 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-iptables-alerter-script\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.317297 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317257 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.317297 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317288 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-sys-fs\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.317424 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317331 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-socket-dir-parent\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.317424 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317359 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-kubelet\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.317531 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/363050d3-1590-4804-b6cc-48cecf994d16-host\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.317630 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317562 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgl7m\" (UniqueName: \"kubernetes.io/projected/363050d3-1590-4804-b6cc-48cecf994d16-kube-api-access-tgl7m\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.317630 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317590 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-host-slash\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.317732 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317632 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.317732 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317665 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtlkp\" (UniqueName: \"kubernetes.io/projected/e301ce06-d062-4d82-9bf1-a91f189bb680-kube-api-access-rtlkp\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.317732 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-cnibin\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.317732 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317729 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-cni-multus\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.317927 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317757 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-registration-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.317927 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317807 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-device-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.317927 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317834 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-cnibin\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.317927 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317875 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:34.317927 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317906 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-system-cni-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.318193 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317948 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-multus-certs\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.318193 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.317977 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-etc-kubernetes\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.318193 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.318026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgw9h\" (UniqueName: \"kubernetes.io/projected/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-kube-api-access-jgw9h\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.318193 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.318049 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c148b74f-c481-4720-bf3d-50bd390af2da-tmp-dir\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.318193 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.318075 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.318193 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.318120 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:34.318193 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.318176 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-cni-binary-copy\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.320175 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.319436 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 06:36:34.336005 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.335988 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bv99j" Sep 15 06:36:34.342434 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.342388 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bv99j" Sep 15 06:36:34.407428 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.407395 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 15 06:36:34.409097 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.409059 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" event={"ID":"cc9a0b5b683b9efc648017b7df9cf169","Type":"ContainerStarted","Data":"e98c3cfadd60d62f088fc3b65edf9fe22c762009787da6cad5447ea7d8705384"} Sep 15 06:36:34.410445 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.410421 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"fd7d5957c3c1631a2eca1f67dcc24ff3","Type":"ContainerStarted","Data":"e2c2b9c830c11f18601b86d52951a7b7b3a0286640b8158ffc7d0fd9a1439f43"} Sep 15 06:36:34.418509 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-multus-certs\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.418621 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418525 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-cnibin\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.418621 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418575 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-cnibin\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.418621 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418572 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-system-cni-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.418778 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-systemd-units\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.418778 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418644 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-system-cni-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.418778 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418655 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysctl-d\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.418778 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418667 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-multus-certs\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.418778 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418675 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-sys\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.418778 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418718 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-var-lib-kubelet\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.418778 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-tuned\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26kwt\" (UniqueName: \"kubernetes.io/projected/8f9c11b3-237a-4501-afcc-8f4c8e30a327-kube-api-access-26kwt\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c148b74f-c481-4720-bf3d-50bd390af2da-tmp-dir\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418860 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-cni-binary-copy\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418908 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-k8s-cni-cncf-io\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-conf-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418941 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-run-netns\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418957 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-etc-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418973 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-ovn\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.418998 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c30094d0-3eb2-4349-b7c0-5f33d52d9e84-agent-certs\") pod \"konnectivity-agent-twhfn\" (UID: \"c30094d0-3eb2-4349-b7c0-5f33d52d9e84\") " pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-system-cni-dir\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.419093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419093 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-netns\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419116 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-node-log\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419168 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c148b74f-c481-4720-bf3d-50bd390af2da-tmp-dir\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-log-socket\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419327 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-system-cni-dir\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419358 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-k8s-cni-cncf-io\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419390 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-conf-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419413 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-run-ovn-kubernetes\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419431 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-ovnkube-script-lib\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419452 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3eadb88e-afda-41c4-836a-c044d4a2628d-kubelet-config\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z9cq\" (UniqueName: \"kubernetes.io/projected/9e225fca-64e1-4eb9-9424-c54b0b77cad6-kube-api-access-4z9cq\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-kubernetes\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-run\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419605 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-socket-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419622 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-cni-binary-copy\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.419751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419665 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-cni-bin\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419719 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419751 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-env-overrides\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419776 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419820 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-cni-binary-copy\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tkkp\" (UniqueName: \"kubernetes.io/projected/71955717-dffc-4ec7-8472-c320b24abff9-kube-api-access-7tkkp\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419971 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-run-netns\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.419990 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-modprobe-d\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420041 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e301ce06-d062-4d82-9bf1-a91f189bb680-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420105 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3eadb88e-afda-41c4-836a-c044d4a2628d-kubelet-config\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420106 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-socket-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.420253 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.420386 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret podName:3eadb88e-afda-41c4-836a-c044d4a2628d nodeName:}" failed. No retries permitted until 2026-09-15 06:36:34.920370347 +0000 UTC m=+2.994747875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret") pod "global-pull-secret-syncer-s4bpp" (UID: "3eadb88e-afda-41c4-836a-c044d4a2628d") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420481 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-host\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420515 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.420543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-registration-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420552 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-device-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420578 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420621 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-etc-kubernetes\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420646 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgw9h\" (UniqueName: \"kubernetes.io/projected/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-kube-api-access-jgw9h\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.420658 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420677 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420700 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-device-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.420749 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:34.920728395 +0000 UTC m=+2.995105941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420732 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-registration-dir\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420813 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-etc-kubernetes\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420812 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.420981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-slash\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421041 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-systemd\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421073 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-ovnkube-config\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421084 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e301ce06-d062-4d82-9bf1-a91f189bb680-cni-binary-copy\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.421627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421109 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/730697a1-01d9-418d-8658-58d762ecdc93-ovn-node-metrics-cert\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5pwf\" (UniqueName: \"kubernetes.io/projected/730697a1-01d9-418d-8658-58d762ecdc93-kube-api-access-l5pwf\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-systemd\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs6zb\" (UniqueName: \"kubernetes.io/projected/c148b74f-c481-4720-bf3d-50bd390af2da-kube-api-access-gs6zb\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421217 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqfdr\" (UniqueName: \"kubernetes.io/projected/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-kube-api-access-fqfdr\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-cni-bin\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421276 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-hostroot\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421329 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-hostroot\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-var-lib-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421388 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysconfig\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421422 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysctl-conf\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421449 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-cni-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421483 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-cni-bin\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421527 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-daemon-config\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c148b74f-c481-4720-bf3d-50bd390af2da-hosts-file\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421684 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-cni-netd\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.422408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421708 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3eadb88e-afda-41c4-836a-c044d4a2628d-dbus\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421734 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/363050d3-1590-4804-b6cc-48cecf994d16-serviceca\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-kubelet\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c30094d0-3eb2-4349-b7c0-5f33d52d9e84-konnectivity-ca\") pod \"konnectivity-agent-twhfn\" (UID: \"c30094d0-3eb2-4349-b7c0-5f33d52d9e84\") " pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421860 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-iptables-alerter-script\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421879 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-cni-dir\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421884 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421927 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-sys-fs\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421965 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-socket-dir-parent\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421973 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.421989 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-kubelet\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422016 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-daemon-config\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422030 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-lib-modules\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422052 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9c11b3-237a-4501-afcc-8f4c8e30a327-tmp\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422085 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-multus-socket-dir-parent\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422092 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-kubelet\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422132 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71955717-dffc-4ec7-8472-c320b24abff9-sys-fs\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.423135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422137 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/363050d3-1590-4804-b6cc-48cecf994d16-host\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422186 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgl7m\" (UniqueName: \"kubernetes.io/projected/363050d3-1590-4804-b6cc-48cecf994d16-kube-api-access-tgl7m\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422213 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/363050d3-1590-4804-b6cc-48cecf994d16-host\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-host-slash\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422252 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtlkp\" (UniqueName: \"kubernetes.io/projected/e301ce06-d062-4d82-9bf1-a91f189bb680-kube-api-access-rtlkp\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422257 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-host-slash\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-cnibin\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422462 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/363050d3-1590-4804-b6cc-48cecf994d16-serviceca\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422534 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3eadb88e-afda-41c4-836a-c044d4a2628d-dbus\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422580 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c148b74f-c481-4720-bf3d-50bd390af2da-hosts-file\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422650 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-cnibin\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-cni-multus\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422731 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-host-var-lib-cni-multus\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.422860 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-iptables-alerter-script\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.423815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.423162 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c30094d0-3eb2-4349-b7c0-5f33d52d9e84-konnectivity-ca\") pod \"konnectivity-agent-twhfn\" (UID: \"c30094d0-3eb2-4349-b7c0-5f33d52d9e84\") " pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:34.424585 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.424559 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c30094d0-3eb2-4349-b7c0-5f33d52d9e84-agent-certs\") pod \"konnectivity-agent-twhfn\" (UID: \"c30094d0-3eb2-4349-b7c0-5f33d52d9e84\") " pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:34.427233 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.427212 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z9cq\" (UniqueName: \"kubernetes.io/projected/9e225fca-64e1-4eb9-9424-c54b0b77cad6-kube-api-access-4z9cq\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:34.431084 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.430797 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:34.431084 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.430825 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:34.431084 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.430839 2581 projected.go:196] Error preparing data for projected volume kube-api-access-k5nht for pod openshift-network-diagnostics/network-check-target-pwmjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:34.431625 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.431574 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht podName:5ed75ee0-83e9-4fe4-8335-43aba5fcba49 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:34.930897441 +0000 UTC m=+3.005274969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k5nht" (UniqueName: "kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht") pod "network-check-target-pwmjh" (UID: "5ed75ee0-83e9-4fe4-8335-43aba5fcba49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:34.431727 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.431704 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tkkp\" (UniqueName: \"kubernetes.io/projected/71955717-dffc-4ec7-8472-c320b24abff9-kube-api-access-7tkkp\") pod \"aws-ebs-csi-driver-node-dmgkf\" (UID: \"71955717-dffc-4ec7-8472-c320b24abff9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.432180 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.432158 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgw9h\" (UniqueName: \"kubernetes.io/projected/e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96-kube-api-access-jgw9h\") pod \"multus-hq4tx\" (UID: \"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96\") " pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.433696 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.433671 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtlkp\" (UniqueName: \"kubernetes.io/projected/e301ce06-d062-4d82-9bf1-a91f189bb680-kube-api-access-rtlkp\") pod \"multus-additional-cni-plugins-n287w\" (UID: \"e301ce06-d062-4d82-9bf1-a91f189bb680\") " pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.433781 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.433660 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs6zb\" (UniqueName: \"kubernetes.io/projected/c148b74f-c481-4720-bf3d-50bd390af2da-kube-api-access-gs6zb\") pod \"node-resolver-mx7sh\" (UID: \"c148b74f-c481-4720-bf3d-50bd390af2da\") " pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.433974 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.433949 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgl7m\" (UniqueName: \"kubernetes.io/projected/363050d3-1590-4804-b6cc-48cecf994d16-kube-api-access-tgl7m\") pod \"node-ca-tjm9m\" (UID: \"363050d3-1590-4804-b6cc-48cecf994d16\") " pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.434212 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.434192 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqfdr\" (UniqueName: \"kubernetes.io/projected/4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2-kube-api-access-fqfdr\") pod \"iptables-alerter-p8pr4\" (UID: \"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2\") " pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.523352 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-kubelet\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.523352 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-lib-modules\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523352 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523328 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9c11b3-237a-4501-afcc-8f4c8e30a327-tmp\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523352 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523353 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-systemd-units\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523376 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysctl-d\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523393 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-kubelet\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523398 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-sys\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523451 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-sys\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523462 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-lib-modules\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523451 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-var-lib-kubelet\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523492 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-var-lib-kubelet\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-tuned\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523532 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26kwt\" (UniqueName: \"kubernetes.io/projected/8f9c11b3-237a-4501-afcc-8f4c8e30a327-kube-api-access-26kwt\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523462 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-systemd-units\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523550 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysctl-d\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523562 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-run-netns\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523597 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-run-netns\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523603 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-etc-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-ovn\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.523659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-node-log\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523660 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-etc-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523685 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-log-socket\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523710 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-run-ovn-kubernetes\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523712 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-ovn\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523734 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-ovnkube-script-lib\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523747 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-node-log\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523763 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-kubernetes\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523784 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-run-ovn-kubernetes\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-run\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523796 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-log-socket\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-cni-bin\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523830 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-run\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523846 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-env-overrides\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523871 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-cni-bin\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523880 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-kubernetes\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-modprobe-d\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-host\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.524423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523948 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-modprobe-d\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523951 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523985 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-host\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.523987 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-slash\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-systemd\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524023 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-ovnkube-config\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524058 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-slash\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524064 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/730697a1-01d9-418d-8658-58d762ecdc93-ovn-node-metrics-cert\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5pwf\" (UniqueName: \"kubernetes.io/projected/730697a1-01d9-418d-8658-58d762ecdc93-kube-api-access-l5pwf\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524110 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-systemd\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-var-lib-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysconfig\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524209 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysctl-conf\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524234 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524257 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-cni-netd\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524324 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-env-overrides\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524326 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-host-cni-netd\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524387 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-var-lib-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524091 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-systemd\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524401 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-ovnkube-script-lib\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524466 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-systemd\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.525772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524494 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/730697a1-01d9-418d-8658-58d762ecdc93-ovnkube-config\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524542 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/730697a1-01d9-418d-8658-58d762ecdc93-run-openvswitch\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.525772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524586 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysconfig\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.525772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.524597 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-sysctl-conf\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.526290 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.526270 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9c11b3-237a-4501-afcc-8f4c8e30a327-tmp\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.526347 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.526314 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f9c11b3-237a-4501-afcc-8f4c8e30a327-etc-tuned\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.526829 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.526811 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/730697a1-01d9-418d-8658-58d762ecdc93-ovn-node-metrics-cert\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.532320 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.532298 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5pwf\" (UniqueName: \"kubernetes.io/projected/730697a1-01d9-418d-8658-58d762ecdc93-kube-api-access-l5pwf\") pod \"ovnkube-node-6p2cg\" (UID: \"730697a1-01d9-418d-8658-58d762ecdc93\") " pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.532797 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.532780 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26kwt\" (UniqueName: \"kubernetes.io/projected/8f9c11b3-237a-4501-afcc-8f4c8e30a327-kube-api-access-26kwt\") pod \"tuned-vvxb9\" (UID: \"8f9c11b3-237a-4501-afcc-8f4c8e30a327\") " pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.605787 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.605744 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:34.612633 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.612613 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mx7sh" Sep 15 06:36:34.613041 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.613016 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30094d0_3eb2_4349_b7c0_5f33d52d9e84.slice/crio-1f9e6c3fde6c5ea7fef2a3dab37699dbd467c1c9b7b6e0940df4b8c380372998 WatchSource:0}: Error finding container 1f9e6c3fde6c5ea7fef2a3dab37699dbd467c1c9b7b6e0940df4b8c380372998: Status 404 returned error can't find the container with id 1f9e6c3fde6c5ea7fef2a3dab37699dbd467c1c9b7b6e0940df4b8c380372998 Sep 15 06:36:34.624542 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.624522 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" Sep 15 06:36:34.628560 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.628540 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tjm9m" Sep 15 06:36:34.630893 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.630836 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc148b74f_c481_4720_bf3d_50bd390af2da.slice/crio-6d095f2bb0f7531a47410235f5d9465c4226c85bc73be6ba51ec79c432d782ce WatchSource:0}: Error finding container 6d095f2bb0f7531a47410235f5d9465c4226c85bc73be6ba51ec79c432d782ce: Status 404 returned error can't find the container with id 6d095f2bb0f7531a47410235f5d9465c4226c85bc73be6ba51ec79c432d782ce Sep 15 06:36:34.633181 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.633137 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71955717_dffc_4ec7_8472_c320b24abff9.slice/crio-108a2072caa5955333761d9e36450429c7014db820f648089eb8bcc594c36bdd WatchSource:0}: Error finding container 108a2072caa5955333761d9e36450429c7014db820f648089eb8bcc594c36bdd: Status 404 returned error can't find the container with id 108a2072caa5955333761d9e36450429c7014db820f648089eb8bcc594c36bdd Sep 15 06:36:34.636619 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.636594 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n287w" Sep 15 06:36:34.636849 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.636828 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod363050d3_1590_4804_b6cc_48cecf994d16.slice/crio-68703a254cfb9f6b21ecb6453719c314fa7bce87be110d62792aa919bdb71332 WatchSource:0}: Error finding container 68703a254cfb9f6b21ecb6453719c314fa7bce87be110d62792aa919bdb71332: Status 404 returned error can't find the container with id 68703a254cfb9f6b21ecb6453719c314fa7bce87be110d62792aa919bdb71332 Sep 15 06:36:34.644093 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.644076 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode301ce06_d062_4d82_9bf1_a91f189bb680.slice/crio-cb3ed45cf92e9fbfe223819061f60c65d8a389aba6c803068e1dc0b602408879 WatchSource:0}: Error finding container cb3ed45cf92e9fbfe223819061f60c65d8a389aba6c803068e1dc0b602408879: Status 404 returned error can't find the container with id cb3ed45cf92e9fbfe223819061f60c65d8a389aba6c803068e1dc0b602408879 Sep 15 06:36:34.644402 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.644382 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p8pr4" Sep 15 06:36:34.652073 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.652055 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hq4tx" Sep 15 06:36:34.654575 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.654557 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac014bb_37f9_44d9_a7b5_fe7a9ae0fae2.slice/crio-937550065d035b77fe624b1a2961d9b533125be5443b2f123d7ca34188b66517 WatchSource:0}: Error finding container 937550065d035b77fe624b1a2961d9b533125be5443b2f123d7ca34188b66517: Status 404 returned error can't find the container with id 937550065d035b77fe624b1a2961d9b533125be5443b2f123d7ca34188b66517 Sep 15 06:36:34.658105 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.658089 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:34.659486 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.659467 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4fd90f4_f3b4_49c5_9fc0_d6352b18ec96.slice/crio-a4147c57080c817618528057ffe07ecdd12db29f5391d2fcc6436021d29a97f6 WatchSource:0}: Error finding container a4147c57080c817618528057ffe07ecdd12db29f5391d2fcc6436021d29a97f6: Status 404 returned error can't find the container with id a4147c57080c817618528057ffe07ecdd12db29f5391d2fcc6436021d29a97f6 Sep 15 06:36:34.662689 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.662671 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" Sep 15 06:36:34.666974 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.666953 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730697a1_01d9_418d_8658_58d762ecdc93.slice/crio-aa7b7e28a80433a93a36f01997fd5a2bdced9705c27a516c34de85d5b1d4e28e WatchSource:0}: Error finding container aa7b7e28a80433a93a36f01997fd5a2bdced9705c27a516c34de85d5b1d4e28e: Status 404 returned error can't find the container with id aa7b7e28a80433a93a36f01997fd5a2bdced9705c27a516c34de85d5b1d4e28e Sep 15 06:36:34.672409 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:36:34.672385 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f9c11b3_237a_4501_afcc_8f4c8e30a327.slice/crio-80f1402b01da095db04030170566e58da6550d4d4fa27be86f3ac2b8da3e6bf0 WatchSource:0}: Error finding container 80f1402b01da095db04030170566e58da6550d4d4fa27be86f3ac2b8da3e6bf0: Status 404 returned error can't find the container with id 80f1402b01da095db04030170566e58da6550d4d4fa27be86f3ac2b8da3e6bf0 Sep 15 06:36:34.707057 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.707035 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:34.927070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.927048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:34.927183 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:34.927079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:34.927231 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.927202 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:34.927231 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.927202 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:34.927295 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.927253 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:35.92723913 +0000 UTC m=+4.001616659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:34.927295 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:34.927266 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret podName:3eadb88e-afda-41c4-836a-c044d4a2628d nodeName:}" failed. No retries permitted until 2026-09-15 06:36:35.927260836 +0000 UTC m=+4.001638363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret") pod "global-pull-secret-syncer-s4bpp" (UID: "3eadb88e-afda-41c4-836a-c044d4a2628d") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:35.027458 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.027434 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:35.027619 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:35.027582 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:35.027619 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:35.027600 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:35.027619 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:35.027611 2581 projected.go:196] Error preparing data for projected volume kube-api-access-k5nht for pod openshift-network-diagnostics/network-check-target-pwmjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:35.027714 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:35.027660 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht podName:5ed75ee0-83e9-4fe4-8335-43aba5fcba49 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:36.027646533 +0000 UTC m=+4.102024074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5nht" (UniqueName: "kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht") pod "network-check-target-pwmjh" (UID: "5ed75ee0-83e9-4fe4-8335-43aba5fcba49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:35.344954 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.344911 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 06:31:34 +0000 UTC" deadline="2028-02-20 15:19:33.281423797 +0000 UTC" Sep 15 06:36:35.344954 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.344953 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12560h42m57.936475334s" Sep 15 06:36:35.434018 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.433983 2581 generic.go:356] "Generic (PLEG): container finished" podID="fd7d5957c3c1631a2eca1f67dcc24ff3" containerID="a3f5163cf4f5990abd3073d8c247dd85c9a4be677d0d0d052fa07bca3b917cb6" exitCode=0 Sep 15 06:36:35.434205 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.434094 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"fd7d5957c3c1631a2eca1f67dcc24ff3","Type":"ContainerDied","Data":"a3f5163cf4f5990abd3073d8c247dd85c9a4be677d0d0d052fa07bca3b917cb6"} Sep 15 06:36:35.451700 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.451667 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" event={"ID":"8f9c11b3-237a-4501-afcc-8f4c8e30a327","Type":"ContainerStarted","Data":"80f1402b01da095db04030170566e58da6550d4d4fa27be86f3ac2b8da3e6bf0"} Sep 15 06:36:35.461978 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.461933 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"aa7b7e28a80433a93a36f01997fd5a2bdced9705c27a516c34de85d5b1d4e28e"} Sep 15 06:36:35.474028 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.473968 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tjm9m" event={"ID":"363050d3-1590-4804-b6cc-48cecf994d16","Type":"ContainerStarted","Data":"68703a254cfb9f6b21ecb6453719c314fa7bce87be110d62792aa919bdb71332"} Sep 15 06:36:35.512957 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.512918 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mx7sh" event={"ID":"c148b74f-c481-4720-bf3d-50bd390af2da","Type":"ContainerStarted","Data":"6d095f2bb0f7531a47410235f5d9465c4226c85bc73be6ba51ec79c432d782ce"} Sep 15 06:36:35.535203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.534341 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hq4tx" event={"ID":"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96","Type":"ContainerStarted","Data":"a4147c57080c817618528057ffe07ecdd12db29f5391d2fcc6436021d29a97f6"} Sep 15 06:36:35.541433 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.541401 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p8pr4" event={"ID":"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2","Type":"ContainerStarted","Data":"937550065d035b77fe624b1a2961d9b533125be5443b2f123d7ca34188b66517"} Sep 15 06:36:35.563385 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.563355 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n287w" event={"ID":"e301ce06-d062-4d82-9bf1-a91f189bb680","Type":"ContainerStarted","Data":"cb3ed45cf92e9fbfe223819061f60c65d8a389aba6c803068e1dc0b602408879"} Sep 15 06:36:35.582669 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.582625 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" event={"ID":"71955717-dffc-4ec7-8472-c320b24abff9","Type":"ContainerStarted","Data":"108a2072caa5955333761d9e36450429c7014db820f648089eb8bcc594c36bdd"} Sep 15 06:36:35.585484 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.585358 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-twhfn" event={"ID":"c30094d0-3eb2-4349-b7c0-5f33d52d9e84","Type":"ContainerStarted","Data":"1f9e6c3fde6c5ea7fef2a3dab37699dbd467c1c9b7b6e0940df4b8c380372998"} Sep 15 06:36:35.938816 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.938723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:35.938816 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:35.938783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:35.939062 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:35.938919 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:35.939062 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:35.938981 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:37.938963206 +0000 UTC m=+6.013340735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:35.939416 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:35.939396 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:35.939486 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:35.939456 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret podName:3eadb88e-afda-41c4-836a-c044d4a2628d nodeName:}" failed. No retries permitted until 2026-09-15 06:36:37.939440112 +0000 UTC m=+6.013817655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret") pod "global-pull-secret-syncer-s4bpp" (UID: "3eadb88e-afda-41c4-836a-c044d4a2628d") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:36.040080 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:36.040039 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:36.040292 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:36.040279 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:36.040350 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:36.040299 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:36.040350 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:36.040311 2581 projected.go:196] Error preparing data for projected volume kube-api-access-k5nht for pod openshift-network-diagnostics/network-check-target-pwmjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:36.040473 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:36.040376 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht podName:5ed75ee0-83e9-4fe4-8335-43aba5fcba49 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:38.040357033 +0000 UTC m=+6.114734563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5nht" (UniqueName: "kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht") pod "network-check-target-pwmjh" (UID: "5ed75ee0-83e9-4fe4-8335-43aba5fcba49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:36.345162 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:36.345081 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 06:31:34 +0000 UTC" deadline="2028-03-28 16:34:40.588528919 +0000 UTC" Sep 15 06:36:36.345162 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:36.345157 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13449h58m4.243390415s" Sep 15 06:36:36.406906 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:36.406877 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:36.407076 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:36.407003 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:36.407135 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:36.407090 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:36.407216 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:36.407195 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:36.407276 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:36.407263 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:36.407357 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:36.407339 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:36.610583 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:36.610544 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"fd7d5957c3c1631a2eca1f67dcc24ff3","Type":"ContainerStarted","Data":"29967e01f2a8d870124ef1a4c0737f4876944e0072f0d603d56537fefabd7b57"} Sep 15 06:36:36.625839 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:36.625780 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" podStartSLOduration=3.625760843 podStartE2EDuration="3.625760843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:36.625264158 +0000 UTC m=+4.699641709" watchObservedRunningTime="2026-09-15 06:36:36.625760843 +0000 UTC m=+4.700138394" Sep 15 06:36:37.957075 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:37.957025 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:37.957562 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:37.957083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:37.957562 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:37.957239 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:37.957562 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:37.957297 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:41.957278404 +0000 UTC m=+10.031655947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:37.957729 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:37.957660 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:37.957729 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:37.957707 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret podName:3eadb88e-afda-41c4-836a-c044d4a2628d nodeName:}" failed. No retries permitted until 2026-09-15 06:36:41.957691765 +0000 UTC m=+10.032069308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret") pod "global-pull-secret-syncer-s4bpp" (UID: "3eadb88e-afda-41c4-836a-c044d4a2628d") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:38.057884 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:38.057846 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:38.058100 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:38.058031 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:38.058100 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:38.058053 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:38.058100 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:38.058066 2581 projected.go:196] Error preparing data for projected volume kube-api-access-k5nht for pod openshift-network-diagnostics/network-check-target-pwmjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:38.058274 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:38.058126 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht podName:5ed75ee0-83e9-4fe4-8335-43aba5fcba49 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:42.058107573 +0000 UTC m=+10.132485115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5nht" (UniqueName: "kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht") pod "network-check-target-pwmjh" (UID: "5ed75ee0-83e9-4fe4-8335-43aba5fcba49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:38.405098 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:38.405010 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:38.405272 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:38.405162 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:38.405562 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:38.405533 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:38.405690 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:38.405650 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:38.405756 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:38.405704 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:38.405815 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:38.405779 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:40.407802 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:40.407672 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:40.408382 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:40.407809 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:40.408382 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:40.407866 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:40.408382 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:40.407953 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:40.408382 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:40.408080 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:40.408382 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:40.408247 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:41.993215 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:41.992924 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:41.993639 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:41.993245 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:41.993639 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:41.993099 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:41.993639 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:41.993369 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret podName:3eadb88e-afda-41c4-836a-c044d4a2628d nodeName:}" failed. No retries permitted until 2026-09-15 06:36:49.993346053 +0000 UTC m=+18.067723586 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret") pod "global-pull-secret-syncer-s4bpp" (UID: "3eadb88e-afda-41c4-836a-c044d4a2628d") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:41.993639 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:41.993374 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:41.993639 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:41.993409 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:49.993398559 +0000 UTC m=+18.067776102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:42.094467 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:42.094418 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:42.094634 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:42.094617 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:42.094692 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:42.094642 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:42.094692 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:42.094655 2581 projected.go:196] Error preparing data for projected volume kube-api-access-k5nht for pod openshift-network-diagnostics/network-check-target-pwmjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:42.094776 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:42.094722 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht podName:5ed75ee0-83e9-4fe4-8335-43aba5fcba49 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:50.094704219 +0000 UTC m=+18.169081770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5nht" (UniqueName: "kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht") pod "network-check-target-pwmjh" (UID: "5ed75ee0-83e9-4fe4-8335-43aba5fcba49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:42.406203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:42.406077 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:42.406203 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:42.406176 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:42.406494 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:42.406480 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:42.406596 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:42.406581 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:42.406661 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:42.406633 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:42.406714 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:42.406683 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:44.406016 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:44.405252 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:44.406016 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:44.405386 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:44.406016 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:44.405760 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:44.406016 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:44.405855 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:44.406016 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:44.405904 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:44.406016 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:44.405973 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:46.405106 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:46.405066 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:46.405572 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:46.405083 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:46.405572 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:46.405223 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:46.405572 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:46.405231 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:46.405572 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:46.405358 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:46.405572 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:46.405433 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:48.405043 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:48.405007 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:48.405573 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:48.405007 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:48.405573 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:48.405173 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:48.405573 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:48.405006 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:48.405573 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:48.405217 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:48.405573 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:48.405310 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:50.054281 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:50.054243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:50.054720 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:50.054288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:50.054720 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.054413 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:50.054720 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.054434 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:50.054720 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.054478 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:37:06.054461298 +0000 UTC m=+34.128838827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:50.054720 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.054504 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret podName:3eadb88e-afda-41c4-836a-c044d4a2628d nodeName:}" failed. No retries permitted until 2026-09-15 06:37:06.054484141 +0000 UTC m=+34.128861672 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret") pod "global-pull-secret-syncer-s4bpp" (UID: "3eadb88e-afda-41c4-836a-c044d4a2628d") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:50.155192 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:50.155130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:50.155365 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.155302 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:50.155365 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.155329 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:50.155365 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.155359 2581 projected.go:196] Error preparing data for projected volume kube-api-access-k5nht for pod openshift-network-diagnostics/network-check-target-pwmjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:50.155537 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.155417 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht podName:5ed75ee0-83e9-4fe4-8335-43aba5fcba49 nodeName:}" failed. No retries permitted until 2026-09-15 06:37:06.155402969 +0000 UTC m=+34.229780501 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5nht" (UniqueName: "kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht") pod "network-check-target-pwmjh" (UID: "5ed75ee0-83e9-4fe4-8335-43aba5fcba49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:50.404879 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:50.404796 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:50.404879 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:50.404862 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:50.404879 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:50.404852 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:50.405124 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.404976 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:50.405124 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.405052 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:50.405275 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:50.405233 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:52.406275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:52.406239 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:52.406670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:52.406346 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:52.406670 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:52.406375 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:52.406670 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:52.406439 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:52.406670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:52.406446 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:52.406670 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:52.406518 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:54.404821 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.404592 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:54.405432 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.404595 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:54.405432 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.404631 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:54.405432 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:54.405128 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:54.405432 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:54.405196 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:54.405432 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:54.404942 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:54.651801 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.651424 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" event={"ID":"cc9a0b5b683b9efc648017b7df9cf169","Type":"ContainerStarted","Data":"012bc70ccdf06ca32a0c1f6c28d42f3a42c90c62a87ae7bf3e9d1f59c223595f"} Sep 15 06:36:54.654891 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.653623 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hq4tx" event={"ID":"e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96","Type":"ContainerStarted","Data":"6ef9bafe1cf8d129e892a68c1b5a04b8fc022985079d0ce24b106f129bf4237d"} Sep 15 06:36:54.658116 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.658086 2581 generic.go:356] "Generic (PLEG): container finished" podID="e301ce06-d062-4d82-9bf1-a91f189bb680" containerID="25e19ef4c83f77ba3d2404e5e4af33ada9702f7dd1ac0d9c67fbfe90fd28d4a1" exitCode=0 Sep 15 06:36:54.658238 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.658177 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n287w" event={"ID":"e301ce06-d062-4d82-9bf1-a91f189bb680","Type":"ContainerDied","Data":"25e19ef4c83f77ba3d2404e5e4af33ada9702f7dd1ac0d9c67fbfe90fd28d4a1"} Sep 15 06:36:54.661053 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.661030 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" event={"ID":"71955717-dffc-4ec7-8472-c320b24abff9","Type":"ContainerStarted","Data":"b6a36e1c692a65f1f75651f532150227e36b0ed72ca0d0761ea6296703d3e7bf"} Sep 15 06:36:54.662662 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.662640 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-twhfn" event={"ID":"c30094d0-3eb2-4349-b7c0-5f33d52d9e84","Type":"ContainerStarted","Data":"e6051da5b14fae50929884d6e26ac0def5ac97aefaef4b538ba5d6b445e883fd"} Sep 15 06:36:54.664373 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.664349 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" event={"ID":"8f9c11b3-237a-4501-afcc-8f4c8e30a327","Type":"ContainerStarted","Data":"98a9f05744c11f567bd489eea7800dcf5d03f81357fd61bcf661afe465d007b2"} Sep 15 06:36:54.667728 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.667708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"041c3182451efd88aa3ffb3a91363d58a610611dc43393ab16348f9331a22c1a"} Sep 15 06:36:54.667827 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.667777 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"2caa0ac527752a820b4acf2bd0e771997a48a710a0ca66d11f5a997c81bc6f91"} Sep 15 06:36:54.667888 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.667827 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"b6b7f2ba0d45d2e39145fd51980d124d430c2b55ea45889380ac2a3a85514410"} Sep 15 06:36:54.667888 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.667874 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"b7b68c635b84d3984e209f12bf87b5cfdfd63d741cf675dc1b6853c469fa499d"} Sep 15 06:36:54.667994 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.667922 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"fb57533ad08c7a6b4391ded9ee79a656de0fdaa8d6d9f0639c2a89194bd2abd8"} Sep 15 06:36:54.667994 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.667969 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"bfafc3cd04ff794ea57f5f56c9a980f7b14ae8181fbfbbb4d8864c22bc69366c"} Sep 15 06:36:54.669426 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.669393 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tjm9m" event={"ID":"363050d3-1590-4804-b6cc-48cecf994d16","Type":"ContainerStarted","Data":"9ecc81218ad057c280004a3908b70c8da3b5ee60bde6dba4ebfdfed06663cec7"} Sep 15 06:36:54.670758 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.670720 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" podStartSLOduration=21.670701714 podStartE2EDuration="21.670701714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:54.670102654 +0000 UTC m=+22.744480205" watchObservedRunningTime="2026-09-15 06:36:54.670701714 +0000 UTC m=+22.745079265" Sep 15 06:36:54.671041 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.671021 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mx7sh" event={"ID":"c148b74f-c481-4720-bf3d-50bd390af2da","Type":"ContainerStarted","Data":"03b313a79fa5c815a4084d3f438ae2d9429a4e644cd95a9f544423714a390cc8"} Sep 15 06:36:54.682772 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.682729 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-twhfn" podStartSLOduration=3.555850476 podStartE2EDuration="22.682717452s" totalImagesPullingTime="19.126866976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:54.682157273 +0000 UTC m=+22.756534820" watchObservedRunningTime="2026-09-15 06:36:54.682717452 +0000 UTC m=+22.757095035" Sep 15 06:36:54.711123 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.711085 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tjm9m" podStartSLOduration=3.5981995209999997 podStartE2EDuration="22.71107262s" totalImagesPullingTime="19.112873099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:54.693700267 +0000 UTC m=+22.768077814" watchObservedRunningTime="2026-09-15 06:36:54.71107262 +0000 UTC m=+22.785450172" Sep 15 06:36:54.725544 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.725499 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hq4tx" podStartSLOduration=3.57971199 podStartE2EDuration="22.725482451s" totalImagesPullingTime="19.145770461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:54.724824049 +0000 UTC m=+22.799201599" watchObservedRunningTime="2026-09-15 06:36:54.725482451 +0000 UTC m=+22.799860001" Sep 15 06:36:54.753941 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.753903 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vvxb9" podStartSLOduration=3.654827875 podStartE2EDuration="22.753890961s" totalImagesPullingTime="19.099063086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:54.738603965 +0000 UTC m=+22.812981515" watchObservedRunningTime="2026-09-15 06:36:54.753890961 +0000 UTC m=+22.828268511" Sep 15 06:36:54.754036 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:54.753989 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mx7sh" podStartSLOduration=3.634978245 podStartE2EDuration="22.75398504s" totalImagesPullingTime="19.119006795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:54.753886427 +0000 UTC m=+22.828263976" watchObservedRunningTime="2026-09-15 06:36:54.75398504 +0000 UTC m=+22.828362590" Sep 15 06:36:55.263257 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:55.263217 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 15 06:36:55.372967 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:55.372860 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T06:36:55.263241335Z","UUID":"2c992fae-8c8a-41f2-876f-d02411a7d3d1","Handler":null,"Name":"","Endpoint":""} Sep 15 06:36:55.374239 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:55.374214 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 15 06:36:55.374239 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:55.374241 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 15 06:36:55.675321 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:55.675282 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" event={"ID":"71955717-dffc-4ec7-8472-c320b24abff9","Type":"ContainerStarted","Data":"f570feff469cb9db6244667a83eb3b0bf7aa9c8f757c97c9d7566338998934a3"} Sep 15 06:36:55.676813 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:55.676764 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p8pr4" event={"ID":"4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2","Type":"ContainerStarted","Data":"fde5c114c66c45eade440493fd655b11e0d055b54da91cdd1c472a3b35c44b1e"} Sep 15 06:36:56.405515 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:56.405490 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:56.405624 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:56.405497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:56.405624 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:56.405595 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:56.405705 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:56.405639 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:56.405705 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:56.405679 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:56.405769 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:56.405734 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:56.684438 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:56.684398 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"e8156e1fedd5a7df22b851956cf7715dc912850e25a3c4fc291d94724bd6225b"} Sep 15 06:36:56.686788 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:56.686726 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" event={"ID":"71955717-dffc-4ec7-8472-c320b24abff9","Type":"ContainerStarted","Data":"1a581c7cca496f4403d16ea126c6709151b6267404381714cad8bde313874129"} Sep 15 06:36:56.703604 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:56.703563 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgkf" podStartSLOduration=3.425816547 podStartE2EDuration="24.70354708s" totalImagesPullingTime="21.277730533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:56.703377065 +0000 UTC m=+24.777754628" watchObservedRunningTime="2026-09-15 06:36:56.70354708 +0000 UTC m=+24.777924631" Sep 15 06:36:56.703844 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:56.703819 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p8pr4" podStartSLOduration=5.60458353 podStartE2EDuration="24.703810629s" totalImagesPullingTime="19.099227099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:55.691419426 +0000 UTC m=+23.765796976" watchObservedRunningTime="2026-09-15 06:36:56.703810629 +0000 UTC m=+24.778188178" Sep 15 06:36:58.245746 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:58.245704 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:58.246315 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:58.246294 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:36:58.404774 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:58.404732 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:36:58.404951 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:58.404747 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:36:58.404951 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:58.404856 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:36:58.404951 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:58.404854 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:36:58.405050 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:58.404954 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:36:58.405050 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:36:58.405034 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:36:59.699434 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:59.699054 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" event={"ID":"730697a1-01d9-418d-8658-58d762ecdc93","Type":"ContainerStarted","Data":"7e179de1c95ffe621fcd1293af347966b3f27c8b2e9fdc8b7e1635feb8817cf4"} Sep 15 06:36:59.711236 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:59.711215 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:59.728851 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:59.728825 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:36:59.738157 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:36:59.738084 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" podStartSLOduration=8.596129138 podStartE2EDuration="27.738063054s" totalImagesPullingTime="19.141933916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:59.736973234 +0000 UTC m=+27.811350802" watchObservedRunningTime="2026-09-15 06:36:59.738063054 +0000 UTC m=+27.812440605" Sep 15 06:37:00.207218 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.207125 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:37:00.207366 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.207253 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 15 06:37:00.207901 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.207885 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-twhfn" Sep 15 06:37:00.405497 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.405474 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:00.405497 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.405498 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:37:00.405713 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:00.405581 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:37:00.405713 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:00.405619 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:37:00.405713 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.405643 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:00.405826 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:00.405727 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:37:00.702763 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.702687 2581 generic.go:356] "Generic (PLEG): container finished" podID="e301ce06-d062-4d82-9bf1-a91f189bb680" containerID="5fa4ebd537edb079512bdd0d6f201a1039e8029863b8984d55f1c38829f63667" exitCode=0 Sep 15 06:37:00.703600 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.702769 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n287w" event={"ID":"e301ce06-d062-4d82-9bf1-a91f189bb680","Type":"ContainerDied","Data":"5fa4ebd537edb079512bdd0d6f201a1039e8029863b8984d55f1c38829f63667"} Sep 15 06:37:00.703600 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.703085 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 15 06:37:00.703600 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.703475 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:37:00.728259 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:00.728236 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:37:01.525729 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:01.525528 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s4bpp"] Sep 15 06:37:01.525890 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:01.525833 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:01.525968 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:01.525940 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:37:01.528528 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:01.528503 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pwmjh"] Sep 15 06:37:01.528662 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:01.528591 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:01.528719 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:01.528665 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:37:01.538166 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:01.538092 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xm2jl"] Sep 15 06:37:01.538275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:01.538198 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:37:01.538333 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:01.538271 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:37:01.705257 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:01.705228 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 15 06:37:02.290196 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:02.290104 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:37:02.712637 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:02.712609 2581 generic.go:356] "Generic (PLEG): container finished" podID="e301ce06-d062-4d82-9bf1-a91f189bb680" containerID="bbf019916ed8f6e479697cc4adbd786dfaea94d575fb0fbd8427885b2ecfd9a1" exitCode=0 Sep 15 06:37:02.712992 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:02.712683 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n287w" event={"ID":"e301ce06-d062-4d82-9bf1-a91f189bb680","Type":"ContainerDied","Data":"bbf019916ed8f6e479697cc4adbd786dfaea94d575fb0fbd8427885b2ecfd9a1"} Sep 15 06:37:03.405236 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:03.405208 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:03.405355 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:03.405209 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:03.405355 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:03.405322 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:37:03.405494 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:03.405209 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:37:03.405494 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:03.405390 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:37:03.405588 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:03.405531 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:37:03.747938 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:03.747869 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:03.822254 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:03.822221 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:04.719438 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:04.719200 2581 generic.go:356] "Generic (PLEG): container finished" podID="e301ce06-d062-4d82-9bf1-a91f189bb680" containerID="3b4c9e9c11f1dbb61370f1a4bde018f88af378b837b3403a68e718a1c75769da" exitCode=0 Sep 15 06:37:04.719438 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:04.719287 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n287w" event={"ID":"e301ce06-d062-4d82-9bf1-a91f189bb680","Type":"ContainerDied","Data":"3b4c9e9c11f1dbb61370f1a4bde018f88af378b837b3403a68e718a1c75769da"} Sep 15 06:37:05.403348 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:05.403319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:05.405269 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:05.405253 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:37:05.405347 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:05.405253 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:05.405428 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:05.405360 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:05.405428 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:05.405358 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:37:05.405520 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:05.405427 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bpp" podUID="3eadb88e-afda-41c4-836a-c044d4a2628d" Sep 15 06:37:05.405562 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:05.405515 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pwmjh" podUID="5ed75ee0-83e9-4fe4-8335-43aba5fcba49" Sep 15 06:37:06.067093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.067065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:06.067093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.067099 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:37:06.067345 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.067219 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:37:06.067345 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.067239 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:37:06.067345 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.067276 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:37:38.067259671 +0000 UTC m=+66.141637216 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:37:06.067345 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.067297 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret podName:3eadb88e-afda-41c4-836a-c044d4a2628d nodeName:}" failed. No retries permitted until 2026-09-15 06:37:38.067282816 +0000 UTC m=+66.141660347 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret") pod "global-pull-secret-syncer-s4bpp" (UID: "3eadb88e-afda-41c4-836a-c044d4a2628d") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:37:06.167815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.167781 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:06.168046 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.167951 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:37:06.168046 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.167980 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:37:06.168046 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.167991 2581 projected.go:196] Error preparing data for projected volume kube-api-access-k5nht for pod openshift-network-diagnostics/network-check-target-pwmjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:37:06.168046 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.168044 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht podName:5ed75ee0-83e9-4fe4-8335-43aba5fcba49 nodeName:}" failed. No retries permitted until 2026-09-15 06:37:38.168027988 +0000 UTC m=+66.242405515 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5nht" (UniqueName: "kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht") pod "network-check-target-pwmjh" (UID: "5ed75ee0-83e9-4fe4-8335-43aba5fcba49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:37:06.763159 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.763110 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeReady" Sep 15 06:37:06.763590 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.763283 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 15 06:37:06.805817 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.805786 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qhvtn"] Sep 15 06:37:06.808804 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.808777 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v5vjb"] Sep 15 06:37:06.808957 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.808916 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.811283 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.811259 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 15 06:37:06.811460 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.811441 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v5vjb" Sep 15 06:37:06.811715 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.811554 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k55p\"" Sep 15 06:37:06.811715 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.811588 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 15 06:37:06.813801 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.813772 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 15 06:37:06.813902 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.813825 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 15 06:37:06.813902 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.813847 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t5mx2\"" Sep 15 06:37:06.813970 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.813951 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 15 06:37:06.817531 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.817486 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v5vjb"] Sep 15 06:37:06.820182 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.820123 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qhvtn"] Sep 15 06:37:06.873550 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.873510 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a563a2a-f128-4058-b119-fe1d206e19fc-tmp-dir\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.873550 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.873549 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c787e62-1891-409b-890d-90dc4095665b-cert\") pod \"ingress-canary-v5vjb\" (UID: \"1c787e62-1891-409b-890d-90dc4095665b\") " pod="openshift-ingress-canary/ingress-canary-v5vjb" Sep 15 06:37:06.873769 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.873591 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t77q\" (UniqueName: \"kubernetes.io/projected/2a563a2a-f128-4058-b119-fe1d206e19fc-kube-api-access-4t77q\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.873769 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.873666 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2a563a2a-f128-4058-b119-fe1d206e19fc-config-volume\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.873769 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.873707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.873769 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.873735 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc2q9\" (UniqueName: \"kubernetes.io/projected/1c787e62-1891-409b-890d-90dc4095665b-kube-api-access-lc2q9\") pod \"ingress-canary-v5vjb\" (UID: \"1c787e62-1891-409b-890d-90dc4095665b\") " pod="openshift-ingress-canary/ingress-canary-v5vjb" Sep 15 06:37:06.940889 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.940792 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:06.974449 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.974423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc2q9\" (UniqueName: \"kubernetes.io/projected/1c787e62-1891-409b-890d-90dc4095665b-kube-api-access-lc2q9\") pod \"ingress-canary-v5vjb\" (UID: \"1c787e62-1891-409b-890d-90dc4095665b\") " pod="openshift-ingress-canary/ingress-canary-v5vjb" Sep 15 06:37:06.974627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.974489 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a563a2a-f128-4058-b119-fe1d206e19fc-tmp-dir\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.974627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.974514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c787e62-1891-409b-890d-90dc4095665b-cert\") pod \"ingress-canary-v5vjb\" (UID: \"1c787e62-1891-409b-890d-90dc4095665b\") " pod="openshift-ingress-canary/ingress-canary-v5vjb" Sep 15 06:37:06.974627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.974536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t77q\" (UniqueName: \"kubernetes.io/projected/2a563a2a-f128-4058-b119-fe1d206e19fc-kube-api-access-4t77q\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.974627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.974592 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2a563a2a-f128-4058-b119-fe1d206e19fc-config-volume\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.974627 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.974624 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.974859 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.974730 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:06.974859 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:06.974796 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:37:07.474776812 +0000 UTC m=+35.549154344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:37:06.974950 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.974891 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a563a2a-f128-4058-b119-fe1d206e19fc-tmp-dir\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.975252 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.975232 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2a563a2a-f128-4058-b119-fe1d206e19fc-config-volume\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.979743 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.979723 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c787e62-1891-409b-890d-90dc4095665b-cert\") pod \"ingress-canary-v5vjb\" (UID: \"1c787e62-1891-409b-890d-90dc4095665b\") " pod="openshift-ingress-canary/ingress-canary-v5vjb" Sep 15 06:37:06.983321 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.983301 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t77q\" (UniqueName: \"kubernetes.io/projected/2a563a2a-f128-4058-b119-fe1d206e19fc-kube-api-access-4t77q\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:06.983810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:06.983783 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc2q9\" (UniqueName: \"kubernetes.io/projected/1c787e62-1891-409b-890d-90dc4095665b-kube-api-access-lc2q9\") pod \"ingress-canary-v5vjb\" (UID: \"1c787e62-1891-409b-890d-90dc4095665b\") " pod="openshift-ingress-canary/ingress-canary-v5vjb" Sep 15 06:37:07.130912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.130881 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v5vjb" Sep 15 06:37:07.404862 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.404829 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:37:07.404862 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.404859 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:07.405059 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.404859 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:07.407850 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.407817 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 06:37:07.407973 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.407881 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 06:37:07.408138 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.408119 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 06:37:07.408829 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.408757 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5bkjm\"" Sep 15 06:37:07.408829 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.408777 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 06:37:07.409083 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.409065 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9fzgn\"" Sep 15 06:37:07.418699 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.418674 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v5vjb"] Sep 15 06:37:07.479693 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:07.479609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:07.479852 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:07.479781 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:07.479914 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:07.479872 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:37:08.479849763 +0000 UTC m=+36.554227294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:37:08.488093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:08.488056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:08.488964 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:08.488217 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:08.488964 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:08.488294 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:37:10.488275599 +0000 UTC m=+38.562653129 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:37:08.838537 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:37:08.838494 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c787e62_1891_409b_890d_90dc4095665b.slice/crio-1fd04bad551d471679426c6f8a9052eed9a4599d742c59b60dac0bf8a3a337ff WatchSource:0}: Error finding container 1fd04bad551d471679426c6f8a9052eed9a4599d742c59b60dac0bf8a3a337ff: Status 404 returned error can't find the container with id 1fd04bad551d471679426c6f8a9052eed9a4599d742c59b60dac0bf8a3a337ff Sep 15 06:37:09.733833 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:09.733648 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v5vjb" event={"ID":"1c787e62-1891-409b-890d-90dc4095665b","Type":"ContainerStarted","Data":"1fd04bad551d471679426c6f8a9052eed9a4599d742c59b60dac0bf8a3a337ff"} Sep 15 06:37:09.737164 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:09.737119 2581 generic.go:356] "Generic (PLEG): container finished" podID="e301ce06-d062-4d82-9bf1-a91f189bb680" containerID="9bb5e761db4d4ab3ddbda5541bb22b0127aa3a5098677561959f767f7a45850f" exitCode=0 Sep 15 06:37:09.737285 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:09.737178 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n287w" event={"ID":"e301ce06-d062-4d82-9bf1-a91f189bb680","Type":"ContainerDied","Data":"9bb5e761db4d4ab3ddbda5541bb22b0127aa3a5098677561959f767f7a45850f"} Sep 15 06:37:10.504446 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:10.504402 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:10.504616 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:10.504509 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:10.504616 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:10.504561 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:37:14.50454761 +0000 UTC m=+42.578925150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:37:10.743867 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:10.743833 2581 generic.go:356] "Generic (PLEG): container finished" podID="e301ce06-d062-4d82-9bf1-a91f189bb680" containerID="dbd4b1a94324f51e55989253e7958815f5e4c1a01eb1b184790ebb97586474d9" exitCode=0 Sep 15 06:37:10.744279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:10.743883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n287w" event={"ID":"e301ce06-d062-4d82-9bf1-a91f189bb680","Type":"ContainerDied","Data":"dbd4b1a94324f51e55989253e7958815f5e4c1a01eb1b184790ebb97586474d9"} Sep 15 06:37:11.267755 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:11.267725 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:11.367568 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:11.367540 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:11.750095 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:11.750059 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n287w" event={"ID":"e301ce06-d062-4d82-9bf1-a91f189bb680","Type":"ContainerStarted","Data":"9cd8664651f31ccbe834fef5acac4dec1986b3a61ce46203b52983af26a6fd8e"} Sep 15 06:37:11.770004 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:11.769961 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n287w" podStartSLOduration=9.247860426999999 podStartE2EDuration="39.769946545s" totalImagesPullingTime="30.522086118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:11.769452666 +0000 UTC m=+39.843830215" watchObservedRunningTime="2026-09-15 06:37:11.769946545 +0000 UTC m=+39.844324095" Sep 15 06:37:11.887594 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:11.887560 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:11.938234 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:11.938135 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:12.754136 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:12.754099 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v5vjb" event={"ID":"1c787e62-1891-409b-890d-90dc4095665b","Type":"ContainerStarted","Data":"bcb8a501de3b810515f74bcb7ca443b6a0f513b7d4aecab85b2346fdae28c3cc"} Sep 15 06:37:12.768668 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:12.768626 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v5vjb" podStartSLOduration=3.950541598 podStartE2EDuration="6.768611135s" totalImagesPullingTime="2.818069537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:12.768075686 +0000 UTC m=+40.842453236" watchObservedRunningTime="2026-09-15 06:37:12.768611135 +0000 UTC m=+40.842988684" Sep 15 06:37:14.262120 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:14.262088 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:14.363397 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:14.363367 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:14.415111 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:14.415085 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:14.534426 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:14.534367 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:14.534537 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:14.534517 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:14.534591 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:14.534581 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:37:22.534566727 +0000 UTC m=+50.608944254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:37:15.667270 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:15.667240 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:15.723913 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:15.723890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:22.588031 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:22.587994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:22.588471 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:22.588207 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:22.588471 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:22.588280 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:37:38.588264714 +0000 UTC m=+66.662642242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:37:26.706272 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:26.706237 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:26.753644 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:26.753611 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:33.724812 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:33.724777 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6p2cg" Sep 15 06:37:33.869236 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:33.869204 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:33.920474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:33.920446 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:36.212550 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:36.212518 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:36.269017 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:36.268989 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:38.092698 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.092655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:38.093086 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.092705 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:37:38.095025 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.095007 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 06:37:38.095095 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.095079 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 06:37:38.102982 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:38.102961 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 15 06:37:38.103079 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:38.103069 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:38:42.103052774 +0000 UTC m=+130.177430301 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : secret "metrics-daemon-secret" not found Sep 15 06:37:38.106384 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.106367 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3eadb88e-afda-41c4-836a-c044d4a2628d-original-pull-secret\") pod \"global-pull-secret-syncer-s4bpp\" (UID: \"3eadb88e-afda-41c4-836a-c044d4a2628d\") " pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:38.193171 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.193128 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:38.196028 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.196012 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 06:37:38.206212 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.206196 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 06:37:38.217571 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.217552 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5nht\" (UniqueName: \"kubernetes.io/projected/5ed75ee0-83e9-4fe4-8335-43aba5fcba49-kube-api-access-k5nht\") pod \"network-check-target-pwmjh\" (UID: \"5ed75ee0-83e9-4fe4-8335-43aba5fcba49\") " pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:38.331222 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.331191 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bpp" Sep 15 06:37:38.338042 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.338014 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9fzgn\"" Sep 15 06:37:38.345930 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.345883 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:38.596256 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.596176 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:37:38.596403 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:38.596291 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:38.596403 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:37:38.596346 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:38:10.596331155 +0000 UTC m=+98.670708697 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:37:38.603721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.603689 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s4bpp"] Sep 15 06:37:38.607081 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:37:38.607055 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eadb88e_afda_41c4_836a_c044d4a2628d.slice/crio-86b59855c4f16b83ee8f7992edba4227652d919107630446b93a357a8619f2a8 WatchSource:0}: Error finding container 86b59855c4f16b83ee8f7992edba4227652d919107630446b93a357a8619f2a8: Status 404 returned error can't find the container with id 86b59855c4f16b83ee8f7992edba4227652d919107630446b93a357a8619f2a8 Sep 15 06:37:38.623089 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.623069 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pwmjh"] Sep 15 06:37:38.625805 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:37:38.625778 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed75ee0_83e9_4fe4_8335_43aba5fcba49.slice/crio-c181d76df42e2ef95b6e97fec9a1aa0cff121aaa7a75861f511e332a9cba4f65 WatchSource:0}: Error finding container c181d76df42e2ef95b6e97fec9a1aa0cff121aaa7a75861f511e332a9cba4f65: Status 404 returned error can't find the container with id c181d76df42e2ef95b6e97fec9a1aa0cff121aaa7a75861f511e332a9cba4f65 Sep 15 06:37:38.816067 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.816013 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pwmjh" event={"ID":"5ed75ee0-83e9-4fe4-8335-43aba5fcba49","Type":"ContainerStarted","Data":"c181d76df42e2ef95b6e97fec9a1aa0cff121aaa7a75861f511e332a9cba4f65"} Sep 15 06:37:38.817077 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:38.817052 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s4bpp" event={"ID":"3eadb88e-afda-41c4-836a-c044d4a2628d","Type":"ContainerStarted","Data":"86b59855c4f16b83ee8f7992edba4227652d919107630446b93a357a8619f2a8"} Sep 15 06:37:40.823220 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:40.823126 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:40.925495 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:40.925425 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:37:43.831577 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:43.831497 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s4bpp" event={"ID":"3eadb88e-afda-41c4-836a-c044d4a2628d","Type":"ContainerStarted","Data":"c86ad6b978249da5f978fe8274250b0a3962aeb3911d6fbe9504e5c07273b346"} Sep 15 06:37:43.833008 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:43.832986 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pwmjh" event={"ID":"5ed75ee0-83e9-4fe4-8335-43aba5fcba49","Type":"ContainerStarted","Data":"cb968fb8f78eeab1143ccfaf371e8775a6f64f931b1f265634b76b9711bc3883"} Sep 15 06:37:43.833125 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:43.833114 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:37:43.845250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:37:43.845090 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s4bpp" podStartSLOduration=66.937801142 podStartE2EDuration="1m11.845075429s" totalImagesPullingTime="4.907274287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:43.844716436 +0000 UTC m=+71.919093987" watchObservedRunningTime="2026-09-15 06:37:43.845075429 +0000 UTC m=+71.919452974" Sep 15 06:38:03.594047 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:03.593929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:38:03.656476 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:03.656446 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:38:10.604203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:10.604130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:38:10.604655 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:38:10.604295 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:38:10.604655 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:38:10.604368 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:39:14.604352405 +0000 UTC m=+162.678729933 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:38:14.838555 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:14.838521 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pwmjh" Sep 15 06:38:14.853944 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:14.853889 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pwmjh" podStartSLOduration=97.96673154 podStartE2EDuration="1m42.853871177s" totalImagesPullingTime="4.887139637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:43.857524659 +0000 UTC m=+71.931902208" watchObservedRunningTime="2026-09-15 06:38:14.853871177 +0000 UTC m=+102.928248726" Sep 15 06:38:42.111211 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:42.111170 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:38:42.111607 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:38:42.111295 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 15 06:38:42.111607 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:38:42.111367 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:44.111347457 +0000 UTC m=+252.185724987 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : secret "metrics-daemon-secret" not found Sep 15 06:38:59.320044 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.320009 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns"] Sep 15 06:38:59.322226 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.322204 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.324682 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.324647 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 15 06:38:59.324794 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.324697 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 15 06:38:59.324794 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.324647 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 15 06:38:59.324794 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.324732 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 15 06:38:59.325733 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.325717 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 15 06:38:59.325814 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.325717 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 15 06:38:59.325814 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.325717 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 15 06:38:59.336006 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.335988 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns"] Sep 15 06:38:59.421794 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.421766 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.421912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.421797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.421912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.421840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-hub\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.421912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.421863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mm4g\" (UniqueName: \"kubernetes.io/projected/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-kube-api-access-9mm4g\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.421912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.421883 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-ca\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.421912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.421897 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.522774 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.522738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.522774 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.522780 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.522889 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.522829 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-hub\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.523001 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.522982 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mm4g\" (UniqueName: \"kubernetes.io/projected/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-kube-api-access-9mm4g\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.523036 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.523016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-ca\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.523083 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.523047 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.524002 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.523975 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.525515 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.525486 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-ca\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.525623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.525486 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-hub\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.525623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.525488 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.525623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.525497 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.531119 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.531096 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mm4g\" (UniqueName: \"kubernetes.io/projected/ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0-kube-api-access-9mm4g\") pod \"cluster-proxy-proxy-agent-c745ff68c-nfmns\" (UID: \"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.641380 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.641302 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:38:59.908786 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:38:59.908713 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns"] Sep 15 06:38:59.911791 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:38:59.911766 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddc5b0a6_b900_4fbc_9f1e_c4e1fd6be0a0.slice/crio-44b290f644b81e9900fa6c6c4a31891bc37ddedb46c33866212094d639d88f39 WatchSource:0}: Error finding container 44b290f644b81e9900fa6c6c4a31891bc37ddedb46c33866212094d639d88f39: Status 404 returned error can't find the container with id 44b290f644b81e9900fa6c6c4a31891bc37ddedb46c33866212094d639d88f39 Sep 15 06:39:00.016991 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:39:00.016958 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" event={"ID":"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0","Type":"ContainerStarted","Data":"44b290f644b81e9900fa6c6c4a31891bc37ddedb46c33866212094d639d88f39"} Sep 15 06:39:04.030799 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:39:04.030763 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" event={"ID":"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0","Type":"ContainerStarted","Data":"eb8ea9ac6702ca97cde905516c302c3dc78d139cdcf54f1ff2955e33ed2feb5c"} Sep 15 06:39:04.031303 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:39:04.030811 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" event={"ID":"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0","Type":"ContainerStarted","Data":"98df4ddd5f24d9b3ea4c9d4f420128de45b641d1b717bfd9bc5ea8da9fb7834a"} Sep 15 06:39:04.031303 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:39:04.030829 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" event={"ID":"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0","Type":"ContainerStarted","Data":"5b81eb88527c153ebd82c49bd6e14172f554d14c152f5561f35c0c9f45687fb1"} Sep 15 06:39:04.048722 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:39:04.048675 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" podStartSLOduration=1.3802810970000001 podStartE2EDuration="5.048660689s" totalImagesPullingTime="3.668379592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:39:04.046944871 +0000 UTC m=+152.121322421" watchObservedRunningTime="2026-09-15 06:39:04.048660689 +0000 UTC m=+152.123038238" Sep 15 06:39:09.824606 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:39:09.824548 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qhvtn" podUID="2a563a2a-f128-4058-b119-fe1d206e19fc" Sep 15 06:39:10.050016 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:39:10.049986 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qhvtn" Sep 15 06:39:10.419033 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:39:10.419000 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:39:14.627226 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:39:14.627187 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:39:14.627633 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:39:14.627309 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:39:14.627633 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:39:14.627369 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls podName:2a563a2a-f128-4058-b119-fe1d206e19fc nodeName:}" failed. No retries permitted until 2026-09-15 06:41:16.627355115 +0000 UTC m=+284.701732647 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls") pod "dns-default-qhvtn" (UID: "2a563a2a-f128-4058-b119-fe1d206e19fc") : secret "dns-default-metrics-tls" not found Sep 15 06:39:24.404995 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:39:24.404901 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:40:07.084873 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:07.084845 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:44.116448 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:44.116394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:40:44.116926 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:44.116559 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 15 06:40:44.116926 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:44.116636 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs podName:9e225fca-64e1-4eb9-9424-c54b0b77cad6 nodeName:}" failed. No retries permitted until 2026-09-15 06:42:46.116619763 +0000 UTC m=+374.190997290 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs") pod "network-metrics-daemon-xm2jl" (UID: "9e225fca-64e1-4eb9-9424-c54b0b77cad6") : secret "metrics-daemon-secret" not found Sep 15 06:40:46.209795 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.209760 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n"] Sep 15 06:40:46.212623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.212605 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:46.215415 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.215393 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kd76g\"" Sep 15 06:40:46.215506 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.215434 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 15 06:40:46.215506 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.215436 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 15 06:40:46.231583 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.231530 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n"] Sep 15 06:40:46.332403 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.332372 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/136a7ac9-282f-44fd-bd26-7284c85bb5df-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:46.332556 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.332414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:46.392968 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.392935 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:46.432882 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.432859 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/136a7ac9-282f-44fd-bd26-7284c85bb5df-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:46.432997 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.432898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:46.433071 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:46.433001 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 15 06:40:46.433071 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:46.433052 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert podName:136a7ac9-282f-44fd-bd26-7284c85bb5df nodeName:}" failed. No retries permitted until 2026-09-15 06:40:46.933040386 +0000 UTC m=+255.007417914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kg2n" (UID: "136a7ac9-282f-44fd-bd26-7284c85bb5df") : secret "networking-console-plugin-cert" not found Sep 15 06:40:46.433543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.433520 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/136a7ac9-282f-44fd-bd26-7284c85bb5df-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:46.493737 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.493714 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:46.937188 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:46.937098 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:46.937315 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:46.937233 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 15 06:40:46.937315 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:46.937306 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert podName:136a7ac9-282f-44fd-bd26-7284c85bb5df nodeName:}" failed. No retries permitted until 2026-09-15 06:40:47.937291122 +0000 UTC m=+256.011668655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kg2n" (UID: "136a7ac9-282f-44fd-bd26-7284c85bb5df") : secret "networking-console-plugin-cert" not found Sep 15 06:40:47.944477 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:47.944424 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:47.944844 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:47.944547 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 15 06:40:47.944844 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:47.944600 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert podName:136a7ac9-282f-44fd-bd26-7284c85bb5df nodeName:}" failed. No retries permitted until 2026-09-15 06:40:49.944587295 +0000 UTC m=+258.018964822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kg2n" (UID: "136a7ac9-282f-44fd-bd26-7284c85bb5df") : secret "networking-console-plugin-cert" not found Sep 15 06:40:49.266912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:49.266879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:49.322685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:49.322650 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:49.959634 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:49.959599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:49.959818 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:49.959719 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 15 06:40:49.959818 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:49.959774 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert podName:136a7ac9-282f-44fd-bd26-7284c85bb5df nodeName:}" failed. No retries permitted until 2026-09-15 06:40:53.95976111 +0000 UTC m=+262.034138637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kg2n" (UID: "136a7ac9-282f-44fd-bd26-7284c85bb5df") : secret "networking-console-plugin-cert" not found Sep 15 06:40:51.749932 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.749899 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gdw2k"] Sep 15 06:40:51.752956 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.752937 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.755447 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.755425 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 15 06:40:51.755555 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.755459 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 15 06:40:51.755555 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.755431 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 15 06:40:51.755555 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.755429 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 15 06:40:51.756486 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.756469 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2dx68\"" Sep 15 06:40:51.762537 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.762518 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gdw2k"] Sep 15 06:40:51.769262 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.769243 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:51.821206 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.821184 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:51.872930 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.872907 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-data-volume\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.873008 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.872933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.873008 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.872959 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.873008 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.872976 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-crio-socket\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.873008 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.872991 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rwz4\" (UniqueName: \"kubernetes.io/projected/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-api-access-2rwz4\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.973995 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.973975 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-data-volume\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.974074 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.973997 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.974074 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.974013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.974074 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.974028 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-crio-socket\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.974074 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.974042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rwz4\" (UniqueName: \"kubernetes.io/projected/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-api-access-2rwz4\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.974297 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:51.974104 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:51.974297 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:51.974132 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:51.974297 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.974137 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-crio-socket\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.974297 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:51.974193 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:52.474173001 +0000 UTC m=+260.548550552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : configmap "kube-rbac-proxy" not found Sep 15 06:40:51.974297 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:51.974214 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:52.474204267 +0000 UTC m=+260.548581802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : secret "insights-runtime-extractor-tls" not found Sep 15 06:40:51.974507 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.974339 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-data-volume\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:51.983784 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:51.983761 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rwz4\" (UniqueName: \"kubernetes.io/projected/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-api-access-2rwz4\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:52.479091 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:52.479054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:52.479091 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:52.479092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:52.479380 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:52.479197 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:52.479380 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:52.479211 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:52.479380 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:52.479263 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:53.479248263 +0000 UTC m=+261.553625791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : secret "insights-runtime-extractor-tls" not found Sep 15 06:40:52.479380 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:52.479278 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:53.479271161 +0000 UTC m=+261.553648689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : configmap "kube-rbac-proxy" not found Sep 15 06:40:53.485971 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:53.485933 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:53.486428 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:53.485989 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:53.486428 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:53.486010 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:53.486428 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:53.486091 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:55.48607056 +0000 UTC m=+263.560448092 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : configmap "kube-rbac-proxy" not found Sep 15 06:40:53.486428 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:53.486094 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:53.486428 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:53.486172 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:55.48613146 +0000 UTC m=+263.560508990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : secret "insights-runtime-extractor-tls" not found Sep 15 06:40:53.924969 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:53.924895 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mx7sh_c148b74f-c481-4720-bf3d-50bd390af2da/dns-node-resolver/0.log" Sep 15 06:40:53.988962 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:53.988933 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:40:53.989087 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:53.989074 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 15 06:40:53.989161 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:53.989133 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert podName:136a7ac9-282f-44fd-bd26-7284c85bb5df nodeName:}" failed. No retries permitted until 2026-09-15 06:41:01.989115645 +0000 UTC m=+270.063493173 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kg2n" (UID: "136a7ac9-282f-44fd-bd26-7284c85bb5df") : secret "networking-console-plugin-cert" not found Sep 15 06:40:54.390345 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.390316 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:54.464473 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.464444 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:54.816656 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.816626 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-twpgb"] Sep 15 06:40:54.819367 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.819352 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:54.821775 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.821756 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 15 06:40:54.822785 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.822759 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 15 06:40:54.822897 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.822798 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7qpmp\"" Sep 15 06:40:54.822897 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.822868 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 15 06:40:54.823035 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.823016 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 15 06:40:54.826635 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.826614 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-twpgb"] Sep 15 06:40:54.849642 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.849618 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:54.896709 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.896689 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-signing-key\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:54.896798 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.896739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-signing-cabundle\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:54.896857 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.896821 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgsd4\" (UniqueName: \"kubernetes.io/projected/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-kube-api-access-bgsd4\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:54.903182 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.903161 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:54.924442 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.924419 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tjm9m_363050d3-1590-4804-b6cc-48cecf994d16/node-ca/0.log" Sep 15 06:40:54.997810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.997792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-signing-cabundle\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:54.997891 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.997835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgsd4\" (UniqueName: \"kubernetes.io/projected/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-kube-api-access-bgsd4\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:54.997891 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.997866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-signing-key\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:54.998445 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:54.998425 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-signing-cabundle\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:55.000409 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:55.000387 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-signing-key\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:55.006335 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:55.006314 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgsd4\" (UniqueName: \"kubernetes.io/projected/ce8dfd64-bb78-4c87-86b7-ba526cf372e5-kube-api-access-bgsd4\") pod \"service-ca-568869fdc5-twpgb\" (UID: \"ce8dfd64-bb78-4c87-86b7-ba526cf372e5\") " pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:55.128703 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:55.128636 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-twpgb" Sep 15 06:40:55.394377 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:55.394308 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-twpgb"] Sep 15 06:40:55.397622 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:40:55.397590 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8dfd64_bb78_4c87_86b7_ba526cf372e5.slice/crio-8a4a2747292824b407596f5d4630d14c0a675c4e90d5e5fec0e8de4ff7b30a09 WatchSource:0}: Error finding container 8a4a2747292824b407596f5d4630d14c0a675c4e90d5e5fec0e8de4ff7b30a09: Status 404 returned error can't find the container with id 8a4a2747292824b407596f5d4630d14c0a675c4e90d5e5fec0e8de4ff7b30a09 Sep 15 06:40:55.501905 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:55.501870 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:55.501905 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:55.501904 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:55.502080 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:55.501995 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:55.502080 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:55.502007 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:55.502080 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:55.502067 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:59.502053463 +0000 UTC m=+267.576430991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : secret "insights-runtime-extractor-tls" not found Sep 15 06:40:55.502080 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:55.502080 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:59.502073937 +0000 UTC m=+267.576451465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : configmap "kube-rbac-proxy" not found Sep 15 06:40:55.711386 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:55.711288 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:55.726774 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:55.726744 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v5vjb_1c787e62-1891-409b-890d-90dc4095665b/serve-healthcheck-canary/0.log" Sep 15 06:40:56.346336 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:56.346306 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-twpgb" event={"ID":"ce8dfd64-bb78-4c87-86b7-ba526cf372e5","Type":"ContainerStarted","Data":"8a4a2747292824b407596f5d4630d14c0a675c4e90d5e5fec0e8de4ff7b30a09"} Sep 15 06:40:58.417980 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:58.417937 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:40:59.355496 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:59.355420 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-twpgb" event={"ID":"ce8dfd64-bb78-4c87-86b7-ba526cf372e5","Type":"ContainerStarted","Data":"84d153d28a6e79bc77575ee44404a215901dddbdc8a5a9cc2852e2fcbfeb3348"} Sep 15 06:40:59.370133 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:59.370087 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-twpgb" podStartSLOduration=1.7679161749999999 podStartE2EDuration="5.370073161s" totalImagesPullingTime="3.602156986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:40:59.369513116 +0000 UTC m=+267.443890667" watchObservedRunningTime="2026-09-15 06:40:59.370073161 +0000 UTC m=+267.444450710" Sep 15 06:40:59.536218 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:59.536178 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:59.536218 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:40:59.536219 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:40:59.536619 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:59.536287 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:59.536619 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:59.536329 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:59.536619 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:59.536342 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:07.536326608 +0000 UTC m=+275.610704137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : configmap "kube-rbac-proxy" not found Sep 15 06:40:59.536619 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:40:59.536365 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:07.536354083 +0000 UTC m=+275.610731611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : secret "insights-runtime-extractor-tls" not found Sep 15 06:41:00.777133 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:00.777089 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:00.933923 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:00.933892 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:01.033971 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:01.033884 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:01.134686 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:01.134602 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:01.189521 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:01.189470 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:01.475442 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:01.475352 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:01.793996 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:01.793964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:02.055932 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:02.055852 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:41:02.056075 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:02.055999 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 15 06:41:02.056075 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:02.056066 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert podName:136a7ac9-282f-44fd-bd26-7284c85bb5df nodeName:}" failed. No retries permitted until 2026-09-15 06:41:18.056050177 +0000 UTC m=+286.130427705 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kg2n" (UID: "136a7ac9-282f-44fd-bd26-7284c85bb5df") : secret "networking-console-plugin-cert" not found Sep 15 06:41:02.676229 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:02.676194 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:02.881589 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:02.881551 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:03.080880 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:03.080841 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:03.276514 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:03.276474 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:03.478997 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:03.478906 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:04.469939 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:04.469905 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:04.524220 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:04.524170 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:04.675648 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:04.675616 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:04.776392 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:04.776363 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:04.947836 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:04.947808 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:05.076181 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:05.076064 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:05.281703 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:05.281671 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:05.689534 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:05.689506 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:06.129755 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:06.129725 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:06.849262 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:06.849228 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:06.950473 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:06.950450 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:07.277133 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:07.277106 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:07.473850 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:07.473816 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:07.598506 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:07.598425 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:41:07.598506 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:07.598455 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:41:07.598719 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:07.598564 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:41:07.598719 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:07.598623 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls podName:d25447d6-a2c6-4d8c-b5bc-b684209f6e02 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:23.598607608 +0000 UTC m=+291.672985136 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdw2k" (UID: "d25447d6-a2c6-4d8c-b5bc-b684209f6e02") : secret "insights-runtime-extractor-tls" not found Sep 15 06:41:07.599044 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:07.599020 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:41:07.676586 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:07.676557 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:08.079557 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:08.079524 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:08.675691 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:08.675655 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:09.079337 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:09.079287 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:09.278221 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:09.278190 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:09.678981 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:09.678951 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:10.270406 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:10.270379 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:10.875733 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:10.875704 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:11.074507 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:11.074470 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:11.284852 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:11.284825 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:11.470249 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:11.470220 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:13.050856 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:13.050820 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qhvtn" podUID="2a563a2a-f128-4058-b119-fe1d206e19fc" Sep 15 06:41:13.400670 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:13.400595 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qhvtn" Sep 15 06:41:14.326478 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:14.326436 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:14.381518 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:14.381494 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:15.609190 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:15.609154 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:15.713270 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:15.713240 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:15.833229 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:15.833202 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:15.888107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:15.888054 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:16.663352 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:16.663309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:41:16.665715 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:16.665688 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a563a2a-f128-4058-b119-fe1d206e19fc-metrics-tls\") pod \"dns-default-qhvtn\" (UID: \"2a563a2a-f128-4058-b119-fe1d206e19fc\") " pod="openshift-dns/dns-default-qhvtn" Sep 15 06:41:16.703945 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:16.703924 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k55p\"" Sep 15 06:41:16.712127 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:16.712109 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qhvtn" Sep 15 06:41:16.981419 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:16.981343 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qhvtn"] Sep 15 06:41:16.984100 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:16.984074 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a563a2a_f128_4058_b119_fe1d206e19fc.slice/crio-1a9eff2727b0ee3ac238b6028850987785089dec06ca554db9f25e11b64d06cc WatchSource:0}: Error finding container 1a9eff2727b0ee3ac238b6028850987785089dec06ca554db9f25e11b64d06cc: Status 404 returned error can't find the container with id 1a9eff2727b0ee3ac238b6028850987785089dec06ca554db9f25e11b64d06cc Sep 15 06:41:17.412106 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:17.412075 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhvtn" event={"ID":"2a563a2a-f128-4058-b119-fe1d206e19fc","Type":"ContainerStarted","Data":"1a9eff2727b0ee3ac238b6028850987785089dec06ca554db9f25e11b64d06cc"} Sep 15 06:41:18.075342 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:18.075311 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:41:18.077795 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:18.077774 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/136a7ac9-282f-44fd-bd26-7284c85bb5df-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kg2n\" (UID: \"136a7ac9-282f-44fd-bd26-7284c85bb5df\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:41:18.320631 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:18.320596 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" Sep 15 06:41:18.754549 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:18.754467 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n"] Sep 15 06:41:18.758269 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:18.758246 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136a7ac9_282f_44fd_bd26_7284c85bb5df.slice/crio-0613eee04480fe39d2c2698bb72b7c1568606db5f8df0e63b65b77953f2ccbe8 WatchSource:0}: Error finding container 0613eee04480fe39d2c2698bb72b7c1568606db5f8df0e63b65b77953f2ccbe8: Status 404 returned error can't find the container with id 0613eee04480fe39d2c2698bb72b7c1568606db5f8df0e63b65b77953f2ccbe8 Sep 15 06:41:19.425685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:19.425640 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhvtn" event={"ID":"2a563a2a-f128-4058-b119-fe1d206e19fc","Type":"ContainerStarted","Data":"5500553ebbb8bef60379952d233f3c30ba371c8c7e27304e551ac21751ad344f"} Sep 15 06:41:19.426158 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:19.425708 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qhvtn" Sep 15 06:41:19.426158 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:19.425734 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhvtn" event={"ID":"2a563a2a-f128-4058-b119-fe1d206e19fc","Type":"ContainerStarted","Data":"7aea6d92669dd594cbaa7850489783fd6c284b81b429b0e84e0f268a6571b130"} Sep 15 06:41:19.427197 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:19.427169 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" event={"ID":"136a7ac9-282f-44fd-bd26-7284c85bb5df","Type":"ContainerStarted","Data":"0613eee04480fe39d2c2698bb72b7c1568606db5f8df0e63b65b77953f2ccbe8"} Sep 15 06:41:19.444260 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:19.444220 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qhvtn" podStartSLOduration=251.886025674 podStartE2EDuration="4m13.444205319s" totalImagesPullingTime="1.558179645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:19.442595698 +0000 UTC m=+287.516973248" watchObservedRunningTime="2026-09-15 06:41:19.444205319 +0000 UTC m=+287.518582868" Sep 15 06:41:20.431158 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.431109 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" event={"ID":"136a7ac9-282f-44fd-bd26-7284c85bb5df","Type":"ContainerStarted","Data":"2de83e8c50de41980112cb7f1c352ce15ebef592217df621fa098632fcaa00d2"} Sep 15 06:41:20.445857 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.445815 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kg2n" podStartSLOduration=33.266768507 podStartE2EDuration="34.445800888s" totalImagesPullingTime="1.179032381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:20.445108961 +0000 UTC m=+288.519486511" watchObservedRunningTime="2026-09-15 06:41:20.445800888 +0000 UTC m=+288.520178485" Sep 15 06:41:20.452512 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.452489 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:20.516353 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.516328 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:20.569918 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.569889 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg"] Sep 15 06:41:20.573371 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.573354 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" Sep 15 06:41:20.575678 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.575659 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 15 06:41:20.575798 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.575778 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lkckv\"" Sep 15 06:41:20.580529 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.580508 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg"] Sep 15 06:41:20.599666 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.599644 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:20.650473 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.650451 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:20.697240 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.697185 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72589117-044c-48cd-8d2c-c6a82a587722-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-gl5lg\" (UID: \"72589117-044c-48cd-8d2c-c6a82a587722\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" Sep 15 06:41:20.798064 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:20.798035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72589117-044c-48cd-8d2c-c6a82a587722-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-gl5lg\" (UID: \"72589117-044c-48cd-8d2c-c6a82a587722\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" Sep 15 06:41:20.798237 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:20.798203 2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 15 06:41:20.798279 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:20.798268 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72589117-044c-48cd-8d2c-c6a82a587722-tls-certificates podName:72589117-044c-48cd-8d2c-c6a82a587722 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:21.298253352 +0000 UTC m=+289.372630885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/72589117-044c-48cd-8d2c-c6a82a587722-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-gl5lg" (UID: "72589117-044c-48cd-8d2c-c6a82a587722") : secret "prometheus-operator-admission-webhook-tls" not found Sep 15 06:41:21.141129 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:21.141098 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:21.303062 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:21.303030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72589117-044c-48cd-8d2c-c6a82a587722-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-gl5lg\" (UID: \"72589117-044c-48cd-8d2c-c6a82a587722\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" Sep 15 06:41:21.305592 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:21.305565 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72589117-044c-48cd-8d2c-c6a82a587722-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-gl5lg\" (UID: \"72589117-044c-48cd-8d2c-c6a82a587722\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" Sep 15 06:41:21.482412 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:21.482335 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" Sep 15 06:41:21.748788 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:21.748758 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg"] Sep 15 06:41:21.752000 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:21.751970 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72589117_044c_48cd_8d2c_c6a82a587722.slice/crio-4f87497fa6e5f7f2c2d55af5a589629eaecebd2bc439442687e830c1035549b8 WatchSource:0}: Error finding container 4f87497fa6e5f7f2c2d55af5a589629eaecebd2bc439442687e830c1035549b8: Status 404 returned error can't find the container with id 4f87497fa6e5f7f2c2d55af5a589629eaecebd2bc439442687e830c1035549b8 Sep 15 06:41:22.437552 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:22.437514 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" event={"ID":"72589117-044c-48cd-8d2c-c6a82a587722","Type":"ContainerStarted","Data":"4f87497fa6e5f7f2c2d55af5a589629eaecebd2bc439442687e830c1035549b8"} Sep 15 06:41:23.442062 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.441966 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" event={"ID":"72589117-044c-48cd-8d2c-c6a82a587722","Type":"ContainerStarted","Data":"807862876db7ef13cb9370d31889515d5b551111e9755eb78051c863b03c7122"} Sep 15 06:41:23.442462 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.442178 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" Sep 15 06:41:23.446838 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.446819 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" Sep 15 06:41:23.456786 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.456748 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-gl5lg" podStartSLOduration=2.2126622400000002 podStartE2EDuration="3.456736845s" totalImagesPullingTime="1.244074605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:23.455679873 +0000 UTC m=+291.530057423" watchObservedRunningTime="2026-09-15 06:41:23.456736845 +0000 UTC m=+291.531114430" Sep 15 06:41:23.591552 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.591518 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:23.619649 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.619585 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:41:23.622119 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.622099 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d25447d6-a2c6-4d8c-b5bc-b684209f6e02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdw2k\" (UID: \"d25447d6-a2c6-4d8c-b5bc-b684209f6e02\") " pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:41:23.656246 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.656216 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:23.730235 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.730169 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:23.861395 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:23.861359 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gdw2k" Sep 15 06:41:24.125968 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:24.125938 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gdw2k"] Sep 15 06:41:24.128892 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:24.128866 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25447d6_a2c6_4d8c_b5bc_b684209f6e02.slice/crio-64d6477189402e2b840478620f4db4c7987691eced09323659cfe9c54b69e3a3 WatchSource:0}: Error finding container 64d6477189402e2b840478620f4db4c7987691eced09323659cfe9c54b69e3a3: Status 404 returned error can't find the container with id 64d6477189402e2b840478620f4db4c7987691eced09323659cfe9c54b69e3a3 Sep 15 06:41:24.446380 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:24.446304 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gdw2k" event={"ID":"d25447d6-a2c6-4d8c-b5bc-b684209f6e02","Type":"ContainerStarted","Data":"efb1b15ff4d1d0a8169f4175384982ae0675daec2997205a2f14c9e15bf2339c"} Sep 15 06:41:24.446380 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:24.446358 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gdw2k" event={"ID":"d25447d6-a2c6-4d8c-b5bc-b684209f6e02","Type":"ContainerStarted","Data":"64d6477189402e2b840478620f4db4c7987691eced09323659cfe9c54b69e3a3"} Sep 15 06:41:25.451743 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:25.451702 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gdw2k" event={"ID":"d25447d6-a2c6-4d8c-b5bc-b684209f6e02","Type":"ContainerStarted","Data":"1ae7c2a4b56f621607620c2c7ebae5a148cc70866d413a1de5c932e5d28b538e"} Sep 15 06:41:27.406307 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:27.406266 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xm2jl" podUID="9e225fca-64e1-4eb9-9424-c54b0b77cad6" Sep 15 06:41:27.458938 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:27.458912 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gdw2k" event={"ID":"d25447d6-a2c6-4d8c-b5bc-b684209f6e02","Type":"ContainerStarted","Data":"00ee130627d7fab2b2a0125fede0bb715cc9b8ad233a35365475ebb7661edf43"} Sep 15 06:41:27.475331 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:27.475289 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gdw2k" podStartSLOduration=34.256185581 podStartE2EDuration="36.475274906s" totalImagesPullingTime="2.219089325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:40:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:27.473491957 +0000 UTC m=+295.547869507" watchObservedRunningTime="2026-09-15 06:41:27.475274906 +0000 UTC m=+295.549652447" Sep 15 06:41:27.758379 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:27.758349 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:28.908287 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:28.908253 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:29.007910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.007872 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:29.199354 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.199269 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:29.396269 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.396235 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:29.433112 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.433086 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qhvtn" Sep 15 06:41:29.578633 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.578602 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:29.642561 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.642520 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" podUID="ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 15 06:41:29.946022 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.945915 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z"] Sep 15 06:41:29.949737 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.949719 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:29.952270 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.952250 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 15 06:41:29.952403 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.952381 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 15 06:41:29.952472 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.952417 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 15 06:41:29.952472 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.952433 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 15 06:41:29.952581 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.952249 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 15 06:41:29.952581 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.952550 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5lhq9\"" Sep 15 06:41:29.958493 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.958463 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z"] Sep 15 06:41:29.959476 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.959451 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:29.959582 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.959489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9615db49-f289-46e6-8f6d-6924c407ba6c-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:29.959582 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.959548 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbrxr\" (UniqueName: \"kubernetes.io/projected/9615db49-f289-46e6-8f6d-6924c407ba6c-kube-api-access-bbrxr\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:29.959582 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.959575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:29.973836 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.973812 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6k8gk"] Sep 15 06:41:29.976862 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.976837 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:29.979398 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.979375 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 15 06:41:29.979497 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.979451 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 15 06:41:29.979608 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.979530 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 15 06:41:29.979678 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:29.979616 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2dkmn\"" Sep 15 06:41:30.060758 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.060721 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-tls\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.060954 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.060771 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.060954 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.060795 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-sys\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.060954 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.060828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.060954 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.060855 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp6z2\" (UniqueName: \"kubernetes.io/projected/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-kube-api-access-qp6z2\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.060954 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.060879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-metrics-client-ca\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.061170 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:30.060954 2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 15 06:41:30.061170 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.060981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-root\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.061170 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:30.061020 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-tls podName:9615db49-f289-46e6-8f6d-6924c407ba6c nodeName:}" failed. No retries permitted until 2026-09-15 06:41:30.56099918 +0000 UTC m=+298.635376707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-pz92z" (UID: "9615db49-f289-46e6-8f6d-6924c407ba6c") : secret "openshift-state-metrics-tls" not found Sep 15 06:41:30.061170 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.061051 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.061170 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.061075 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9615db49-f289-46e6-8f6d-6924c407ba6c-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.061170 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.061123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbrxr\" (UniqueName: \"kubernetes.io/projected/9615db49-f289-46e6-8f6d-6924c407ba6c-kube-api-access-bbrxr\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.061170 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.061163 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-accelerators-collector-config\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.061502 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.061186 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-textfile\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.061502 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.061208 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-wtmp\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.061834 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.061817 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9615db49-f289-46e6-8f6d-6924c407ba6c-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.064008 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.063986 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.071306 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.071278 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbrxr\" (UniqueName: \"kubernetes.io/projected/9615db49-f289-46e6-8f6d-6924c407ba6c-kube-api-access-bbrxr\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.162010 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.161975 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp6z2\" (UniqueName: \"kubernetes.io/projected/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-kube-api-access-qp6z2\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162010 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162017 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-metrics-client-ca\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162258 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162039 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-root\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162258 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162091 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-accelerators-collector-config\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162258 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-textfile\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162258 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162138 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-wtmp\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162258 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162204 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-tls\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162258 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162234 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162548 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162259 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-sys\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162548 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162339 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-sys\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162548 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:30.162440 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 15 06:41:30.162548 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:30.162505 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-tls podName:bd77cbce-b2ff-48d9-a6c1-f41d3030622a nodeName:}" failed. No retries permitted until 2026-09-15 06:41:30.662487761 +0000 UTC m=+298.736865296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-tls") pod "node-exporter-6k8gk" (UID: "bd77cbce-b2ff-48d9-a6c1-f41d3030622a") : secret "node-exporter-tls" not found Sep 15 06:41:30.162749 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-textfile\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162749 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162673 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-root\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162749 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162710 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-metrics-client-ca\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162902 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162811 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-accelerators-collector-config\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.162902 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.162846 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-wtmp\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.164968 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.164938 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.170074 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.170048 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp6z2\" (UniqueName: \"kubernetes.io/projected/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-kube-api-access-qp6z2\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.565402 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.565364 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.567983 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.567952 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9615db49-f289-46e6-8f6d-6924c407ba6c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pz92z\" (UID: \"9615db49-f289-46e6-8f6d-6924c407ba6c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.666252 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.666223 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-tls\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.668521 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.668504 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd77cbce-b2ff-48d9-a6c1-f41d3030622a-node-exporter-tls\") pod \"node-exporter-6k8gk\" (UID: \"bd77cbce-b2ff-48d9-a6c1-f41d3030622a\") " pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:30.860370 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.860289 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" Sep 15 06:41:30.885980 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:30.885951 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6k8gk" Sep 15 06:41:31.028856 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.028825 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:41:31.033231 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.033215 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.035743 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.035722 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 15 06:41:31.035836 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.035785 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 15 06:41:31.036045 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.036024 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 15 06:41:31.036176 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.036051 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 15 06:41:31.036176 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.036068 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 15 06:41:31.036176 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.036077 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 15 06:41:31.036409 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.036381 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 15 06:41:31.036470 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.036409 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 15 06:41:31.036522 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.036472 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2bf5d\"" Sep 15 06:41:31.036522 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.036496 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 15 06:41:31.049043 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.049021 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:41:31.069548 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069522 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069633 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-web-config\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069633 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069578 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069717 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069696 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-config-volume\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069755 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069788 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069773 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069820 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069799 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069866 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069830 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069892 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-config-out\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069958 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.069958 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069930 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.070030 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069951 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.070030 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.069994 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7tnj\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-kube-api-access-d7tnj\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.135735 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.135709 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z"] Sep 15 06:41:31.138360 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:31.138332 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9615db49_f289_46e6_8f6d_6924c407ba6c.slice/crio-997f36339a91dbd3c659f30f2844a637ea96b62e4ee1eafbee485de5c4040c99 WatchSource:0}: Error finding container 997f36339a91dbd3c659f30f2844a637ea96b62e4ee1eafbee485de5c4040c99: Status 404 returned error can't find the container with id 997f36339a91dbd3c659f30f2844a637ea96b62e4ee1eafbee485de5c4040c99 Sep 15 06:41:31.171065 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.171043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-config-volume\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.171180 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.171073 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.171180 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.171093 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.171180 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.171109 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.171180 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.171126 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172054 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.171853 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172221 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.172196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-config-out\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172406 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.172256 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172482 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.172425 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172482 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.172457 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172577 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.172493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tnj\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-kube-api-access-d7tnj\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172577 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.172529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172686 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.172666 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-web-config\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.172969 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.172724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.174954 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.174925 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-config-volume\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.175038 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.175000 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-config-out\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.175172 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:31.175134 2581 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 15 06:41:31.175230 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:31.175224 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls podName:238018be-52e8-46b5-96a4-4234958c24b9 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:31.675205558 +0000 UTC m=+299.749583102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "238018be-52e8-46b5-96a4-4234958c24b9") : secret "alertmanager-main-tls" not found Sep 15 06:41:31.175573 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.175553 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.177070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.176585 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.177070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.176920 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.178178 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.178129 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.179379 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.178784 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-web-config\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.179379 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.178898 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.182677 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.182650 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.183062 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.183027 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.187009 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.186988 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7tnj\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-kube-api-access-d7tnj\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.472122 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.472035 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6k8gk" event={"ID":"bd77cbce-b2ff-48d9-a6c1-f41d3030622a","Type":"ContainerStarted","Data":"3b42ffe26d073499134818392157769b0fe36a0acf4185839cd0d27c5156d3e4"} Sep 15 06:41:31.474485 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.474458 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" event={"ID":"9615db49-f289-46e6-8f6d-6924c407ba6c","Type":"ContainerStarted","Data":"f618224d7dac9eb6480c9642751f8f7258e330146ea13cc10b95c493e65151a1"} Sep 15 06:41:31.474606 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.474513 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" event={"ID":"9615db49-f289-46e6-8f6d-6924c407ba6c","Type":"ContainerStarted","Data":"11b776b06e095199095f62f46784210549c66c9f84cfe8a7e2c500d665a201b7"} Sep 15 06:41:31.474606 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.474544 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" event={"ID":"9615db49-f289-46e6-8f6d-6924c407ba6c","Type":"ContainerStarted","Data":"997f36339a91dbd3c659f30f2844a637ea96b62e4ee1eafbee485de5c4040c99"} Sep 15 06:41:31.676047 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:31.676013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:31.676227 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:31.676157 2581 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 15 06:41:31.676227 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:41:31.676225 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls podName:238018be-52e8-46b5-96a4-4234958c24b9 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:32.676208281 +0000 UTC m=+300.750585811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "238018be-52e8-46b5-96a4-4234958c24b9") : secret "alertmanager-main-tls" not found Sep 15 06:41:32.327613 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.327587 2581 kubelet.go:1648] "Image garbage collection succeeded" Sep 15 06:41:32.478902 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.478740 2581 generic.go:356] "Generic (PLEG): container finished" podID="bd77cbce-b2ff-48d9-a6c1-f41d3030622a" containerID="0de6a5780bc594592ded5d5daa4e347cdfc4b6b7b94e8fcad9c7b1845796cc0f" exitCode=0 Sep 15 06:41:32.478902 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.478808 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6k8gk" event={"ID":"bd77cbce-b2ff-48d9-a6c1-f41d3030622a","Type":"ContainerDied","Data":"0de6a5780bc594592ded5d5daa4e347cdfc4b6b7b94e8fcad9c7b1845796cc0f"} Sep 15 06:41:32.684086 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.684051 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:32.686524 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.686501 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:32.845246 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.845177 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2bf5d\"" Sep 15 06:41:32.854076 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.854051 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:41:32.958016 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.957984 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-67d6476fb-8vnwr"] Sep 15 06:41:32.961461 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.961446 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:32.963937 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.963912 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 15 06:41:32.963937 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.963926 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 15 06:41:32.964119 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.963954 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 15 06:41:32.964119 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.963974 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8f8c2\"" Sep 15 06:41:32.964119 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.963983 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5vigppmk6u6va\"" Sep 15 06:41:32.964119 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.963999 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 15 06:41:32.964119 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.963929 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 15 06:41:32.972769 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.972749 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67d6476fb-8vnwr"] Sep 15 06:41:32.986475 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.986449 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:32.986584 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.986485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d726025-6fe8-44f7-87de-a54afde16e3e-metrics-client-ca\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:32.986584 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.986512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:32.986584 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.986567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phk2f\" (UniqueName: \"kubernetes.io/projected/1d726025-6fe8-44f7-87de-a54afde16e3e-kube-api-access-phk2f\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:32.986705 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.986640 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-grpc-tls\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:32.986737 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.986701 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:32.986770 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.986741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-tls\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:32.986808 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:32.986767 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.088091 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.088049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-grpc-tls\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.088264 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.088101 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.088264 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.088224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-tls\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.088379 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.088265 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.088379 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.088324 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.088379 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.088360 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d726025-6fe8-44f7-87de-a54afde16e3e-metrics-client-ca\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.088532 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.088405 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.088532 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.088441 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phk2f\" (UniqueName: \"kubernetes.io/projected/1d726025-6fe8-44f7-87de-a54afde16e3e-kube-api-access-phk2f\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.089222 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.089170 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d726025-6fe8-44f7-87de-a54afde16e3e-metrics-client-ca\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.091293 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.091251 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.091610 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.091588 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.091729 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.091690 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.091849 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.091825 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.091890 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.091838 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-thanos-querier-tls\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.091923 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.091894 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1d726025-6fe8-44f7-87de-a54afde16e3e-secret-grpc-tls\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.095661 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.095619 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phk2f\" (UniqueName: \"kubernetes.io/projected/1d726025-6fe8-44f7-87de-a54afde16e3e-kube-api-access-phk2f\") pod \"thanos-querier-67d6476fb-8vnwr\" (UID: \"1d726025-6fe8-44f7-87de-a54afde16e3e\") " pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.135933 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.135905 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:41:33.139506 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:33.139484 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238018be_52e8_46b5_96a4_4234958c24b9.slice/crio-2b6fb96a82b2b4a1828be02f9dd3234b59ee41fbc57597a3c7f87f646bdb8ec9 WatchSource:0}: Error finding container 2b6fb96a82b2b4a1828be02f9dd3234b59ee41fbc57597a3c7f87f646bdb8ec9: Status 404 returned error can't find the container with id 2b6fb96a82b2b4a1828be02f9dd3234b59ee41fbc57597a3c7f87f646bdb8ec9 Sep 15 06:41:33.270848 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.270810 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:33.483571 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.483475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerStarted","Data":"2b6fb96a82b2b4a1828be02f9dd3234b59ee41fbc57597a3c7f87f646bdb8ec9"} Sep 15 06:41:33.485582 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.485559 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6k8gk" event={"ID":"bd77cbce-b2ff-48d9-a6c1-f41d3030622a","Type":"ContainerStarted","Data":"b977a27cca3f80ad2493aee123c6c83a745c42d312294a36032a1beaedc793ab"} Sep 15 06:41:33.485681 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.485601 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6k8gk" event={"ID":"bd77cbce-b2ff-48d9-a6c1-f41d3030622a","Type":"ContainerStarted","Data":"74f273d65697341a3762573f0b63445226e5126a3af38ce5c6cd5b4435727b04"} Sep 15 06:41:33.503048 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.503008 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6k8gk" podStartSLOduration=3.40177457 podStartE2EDuration="4.502993881s" totalImagesPullingTime="1.032984769s" totalInitContainerRuntime="68.234542ms" isStatefulPod=true podCreationTimestamp="2026-09-15 06:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:33.501522729 +0000 UTC m=+301.575900280" watchObservedRunningTime="2026-09-15 06:41:33.502993881 +0000 UTC m=+301.577371432" Sep 15 06:41:33.540837 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.540814 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67d6476fb-8vnwr"] Sep 15 06:41:33.544114 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:33.544091 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d726025_6fe8_44f7_87de_a54afde16e3e.slice/crio-9e05749bdd330fdf65c6f98a71876de30ccd40bec2ac1af28e2231c9d6252eb1 WatchSource:0}: Error finding container 9e05749bdd330fdf65c6f98a71876de30ccd40bec2ac1af28e2231c9d6252eb1: Status 404 returned error can't find the container with id 9e05749bdd330fdf65c6f98a71876de30ccd40bec2ac1af28e2231c9d6252eb1 Sep 15 06:41:33.545999 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:33.545983 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 06:41:34.024081 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.024045 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:34.263662 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.263635 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-559d484c58-c2mgc"] Sep 15 06:41:34.266563 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.266544 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.270051 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.269918 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 15 06:41:34.270051 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.269999 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nws5s\"" Sep 15 06:41:34.270413 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.270257 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 15 06:41:34.270413 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.270404 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 15 06:41:34.270554 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.270437 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 15 06:41:34.270554 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.270407 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7iqvmeguoevt0\"" Sep 15 06:41:34.276390 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.276343 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-559d484c58-c2mgc"] Sep 15 06:41:34.299640 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.299618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-secret-metrics-server-tls\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.299752 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.299646 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/537f65bf-b80e-4e46-8270-131d908e76bd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.299752 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.299667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/537f65bf-b80e-4e46-8270-131d908e76bd-metrics-server-audit-profiles\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.299752 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.299681 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxtml\" (UniqueName: \"kubernetes.io/projected/537f65bf-b80e-4e46-8270-131d908e76bd-kube-api-access-gxtml\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.299752 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.299700 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-secret-metrics-server-client-certs\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.299752 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.299718 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/537f65bf-b80e-4e46-8270-131d908e76bd-audit-log\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.299752 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.299749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-client-ca-bundle\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.400746 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.400715 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-secret-metrics-server-tls\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.400925 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.400758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/537f65bf-b80e-4e46-8270-131d908e76bd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.400925 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.400798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/537f65bf-b80e-4e46-8270-131d908e76bd-metrics-server-audit-profiles\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.400925 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.400823 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxtml\" (UniqueName: \"kubernetes.io/projected/537f65bf-b80e-4e46-8270-131d908e76bd-kube-api-access-gxtml\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.400925 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.400843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-secret-metrics-server-client-certs\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.400925 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.400866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/537f65bf-b80e-4e46-8270-131d908e76bd-audit-log\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.400925 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.400918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-client-ca-bundle\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.401775 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.401656 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/537f65bf-b80e-4e46-8270-131d908e76bd-audit-log\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.402544 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.402471 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/537f65bf-b80e-4e46-8270-131d908e76bd-metrics-server-audit-profiles\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.402544 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.402506 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/537f65bf-b80e-4e46-8270-131d908e76bd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.403707 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.403666 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-client-ca-bundle\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.404372 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.404349 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-secret-metrics-server-tls\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.404609 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.404590 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/537f65bf-b80e-4e46-8270-131d908e76bd-secret-metrics-server-client-certs\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.410326 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.410287 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxtml\" (UniqueName: \"kubernetes.io/projected/537f65bf-b80e-4e46-8270-131d908e76bd-kube-api-access-gxtml\") pod \"metrics-server-559d484c58-c2mgc\" (UID: \"537f65bf-b80e-4e46-8270-131d908e76bd\") " pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.491708 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.491668 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" event={"ID":"1d726025-6fe8-44f7-87de-a54afde16e3e","Type":"ContainerStarted","Data":"9e05749bdd330fdf65c6f98a71876de30ccd40bec2ac1af28e2231c9d6252eb1"} Sep 15 06:41:34.494818 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.494735 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" event={"ID":"9615db49-f289-46e6-8f6d-6924c407ba6c","Type":"ContainerStarted","Data":"2acf401fc50329645fa448320ec27059b537d3f98744f04684ee766a244c8300"} Sep 15 06:41:34.511754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.511702 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pz92z" podStartSLOduration=2.540942089 podStartE2EDuration="5.511683669s" totalImagesPullingTime="2.97074158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:34.510729043 +0000 UTC m=+302.585106595" watchObservedRunningTime="2026-09-15 06:41:34.511683669 +0000 UTC m=+302.586061221" Sep 15 06:41:34.590793 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.590719 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:34.760250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.760220 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:34.879191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.879087 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-559d484c58-c2mgc"] Sep 15 06:41:34.881765 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:34.881739 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:34.882527 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:34.882497 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537f65bf_b80e_4e46_8270_131d908e76bd.slice/crio-835f564a3d3eb1076173c77e14cc0e58e5bdeb6d09f234cd065cff47dc21a86a WatchSource:0}: Error finding container 835f564a3d3eb1076173c77e14cc0e58e5bdeb6d09f234cd065cff47dc21a86a: Status 404 returned error can't find the container with id 835f564a3d3eb1076173c77e14cc0e58e5bdeb6d09f234cd065cff47dc21a86a Sep 15 06:41:35.122263 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:35.122232 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:35.499737 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:35.499704 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" event={"ID":"537f65bf-b80e-4e46-8270-131d908e76bd","Type":"ContainerStarted","Data":"835f564a3d3eb1076173c77e14cc0e58e5bdeb6d09f234cd065cff47dc21a86a"} Sep 15 06:41:35.501408 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:35.501383 2581 generic.go:356] "Generic (PLEG): container finished" podID="238018be-52e8-46b5-96a4-4234958c24b9" containerID="fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601" exitCode=0 Sep 15 06:41:35.501538 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:35.501473 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerDied","Data":"fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601"} Sep 15 06:41:36.187011 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:36.186918 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:36.508932 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:36.508754 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" event={"ID":"1d726025-6fe8-44f7-87de-a54afde16e3e","Type":"ContainerStarted","Data":"21169d93e96fb7869d000df2517b8b326a9bd242cf3623097daab3372e71ec57"} Sep 15 06:41:36.508932 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:36.508844 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" event={"ID":"1d726025-6fe8-44f7-87de-a54afde16e3e","Type":"ContainerStarted","Data":"4b6d7932feddaa288dba5fc7234e23587242f45e42b05e5b8f816dcbb0b8c729"} Sep 15 06:41:36.508932 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:36.508885 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" event={"ID":"1d726025-6fe8-44f7-87de-a54afde16e3e","Type":"ContainerStarted","Data":"73a96855d46b10da45bdbc9a4c02c06de31eaecb54d44e57075fd804ad58c7ae"} Sep 15 06:41:37.516989 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.516926 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerStarted","Data":"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5"} Sep 15 06:41:37.517333 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.517008 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerStarted","Data":"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da"} Sep 15 06:41:37.517333 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.517054 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerStarted","Data":"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be"} Sep 15 06:41:37.517333 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.517097 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerStarted","Data":"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d"} Sep 15 06:41:37.519999 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.519974 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" event={"ID":"1d726025-6fe8-44f7-87de-a54afde16e3e","Type":"ContainerStarted","Data":"7648bd5576521fdddbdf7c83687fa96c59daa2db5a53ee7ef0820cc430324d65"} Sep 15 06:41:37.520175 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.520042 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" event={"ID":"1d726025-6fe8-44f7-87de-a54afde16e3e","Type":"ContainerStarted","Data":"445fc0e89799a0ae48c597f57dbfe981da1f140485be637b9a7731b8e8c0bc0a"} Sep 15 06:41:37.520175 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.520085 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" event={"ID":"1d726025-6fe8-44f7-87de-a54afde16e3e","Type":"ContainerStarted","Data":"84051d31f9999e796e4c801894cb46d10289c95a434e3790ff3e79ec50947e87"} Sep 15 06:41:37.520393 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.520373 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:37.522020 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.522002 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" event={"ID":"537f65bf-b80e-4e46-8270-131d908e76bd","Type":"ContainerStarted","Data":"cb8e13d36ccb33150a1fa958c540684ddb2df8e3aa8ae1e45df780ceedb1fe72"} Sep 15 06:41:37.542197 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.542134 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" podStartSLOduration=2.036176528 podStartE2EDuration="5.542119767s" totalImagesPullingTime="3.505943239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:37.540729902 +0000 UTC m=+305.615107453" watchObservedRunningTime="2026-09-15 06:41:37.542119767 +0000 UTC m=+305.616497317" Sep 15 06:41:37.558207 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.557268 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" podStartSLOduration=1.24698466 podStartE2EDuration="3.55725532s" totalImagesPullingTime="2.31027066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:37.557213202 +0000 UTC m=+305.631590755" watchObservedRunningTime="2026-09-15 06:41:37.55725532 +0000 UTC m=+305.631632848" Sep 15 06:41:37.873611 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.873534 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:37.973433 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:37.973400 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:38.405225 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:38.405192 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:41:38.530915 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:38.530883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerStarted","Data":"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042"} Sep 15 06:41:38.531275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:38.530962 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerStarted","Data":"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008"} Sep 15 06:41:38.562354 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:38.562311 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.385630757 podStartE2EDuration="7.562297105s" totalImagesPullingTime="3.159209775s" totalInitContainerRuntime="17.456573ms" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:38.560526537 +0000 UTC m=+306.634904089" watchObservedRunningTime="2026-09-15 06:41:38.562297105 +0000 UTC m=+306.636674655" Sep 15 06:41:39.643179 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:39.643126 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" podUID="ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 15 06:41:41.434114 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:41.434086 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:43.536893 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:43.536866 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-67d6476fb-8vnwr" Sep 15 06:41:43.547297 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:43.547275 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:43.648787 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:43.648755 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:43.805855 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:43.805773 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:43.856563 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:43.856534 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:44.323023 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:44.322992 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:44.379661 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:44.379636 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:47.609476 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:47.609442 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:47.710342 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:47.710313 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:48.602285 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.602255 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-688d95dd96-8bl7t"] Sep 15 06:41:48.608666 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.608641 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.612194 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.612165 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 15 06:41:48.612194 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.612177 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 15 06:41:48.612657 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.612295 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 15 06:41:48.612657 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.612295 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 15 06:41:48.612657 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.612435 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 15 06:41:48.612657 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.612470 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 15 06:41:48.612657 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.612646 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ksjg5\"" Sep 15 06:41:48.612866 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.612690 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 15 06:41:48.616684 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.616666 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 15 06:41:48.617686 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.617664 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688d95dd96-8bl7t"] Sep 15 06:41:48.711973 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.711952 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-trusted-ca-bundle\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.712092 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.711981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-serving-cert\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.712092 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.712001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-service-ca\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.712092 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.712082 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-oauth-serving-cert\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.712227 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.712171 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-config\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.712227 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.712197 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htnkp\" (UniqueName: \"kubernetes.io/projected/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-kube-api-access-htnkp\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.712227 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.712216 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-oauth-config\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.813337 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.813309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-oauth-serving-cert\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.813451 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.813355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-config\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.813451 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.813378 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htnkp\" (UniqueName: \"kubernetes.io/projected/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-kube-api-access-htnkp\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.813451 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.813409 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-oauth-config\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.813616 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.813479 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-trusted-ca-bundle\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.813616 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.813510 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-serving-cert\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.813616 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.813538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-service-ca\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.814125 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.814103 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-oauth-serving-cert\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.814239 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.814106 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-config\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.814239 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.814219 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-service-ca\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.814355 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.814343 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-trusted-ca-bundle\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.816167 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.816122 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-serving-cert\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.816259 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.816178 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-oauth-config\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.823694 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.823674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htnkp\" (UniqueName: \"kubernetes.io/projected/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-kube-api-access-htnkp\") pod \"console-688d95dd96-8bl7t\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:48.918913 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:48.918849 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:49.188529 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:49.188456 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688d95dd96-8bl7t"] Sep 15 06:41:49.191361 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:41:49.191338 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842fbd95_0a52_42eb_9ae4_7ac9e8a64fb6.slice/crio-74c7a6334897f94099e31dcb3bfb30719a7b4a8a4572ad29487a35d77221d7c0 WatchSource:0}: Error finding container 74c7a6334897f94099e31dcb3bfb30719a7b4a8a4572ad29487a35d77221d7c0: Status 404 returned error can't find the container with id 74c7a6334897f94099e31dcb3bfb30719a7b4a8a4572ad29487a35d77221d7c0 Sep 15 06:41:49.569233 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:49.569197 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688d95dd96-8bl7t" event={"ID":"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6","Type":"ContainerStarted","Data":"74c7a6334897f94099e31dcb3bfb30719a7b4a8a4572ad29487a35d77221d7c0"} Sep 15 06:41:49.642909 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:49.642875 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" podUID="ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 15 06:41:49.643263 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:49.642944 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" Sep 15 06:41:49.643432 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:49.643404 2581 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"eb8ea9ac6702ca97cde905516c302c3dc78d139cdcf54f1ff2955e33ed2feb5c"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" containerMessage="Container service-proxy failed liveness probe, will be restarted" Sep 15 06:41:49.643469 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:49.643453 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" podUID="ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0" containerName="service-proxy" containerID="cri-o://eb8ea9ac6702ca97cde905516c302c3dc78d139cdcf54f1ff2955e33ed2feb5c" gracePeriod=30 Sep 15 06:41:50.111397 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:50.111314 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:50.209642 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:50.209608 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:50.411447 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:50.411361 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:50.512790 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:50.512732 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:50.576021 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:50.575989 2581 generic.go:356] "Generic (PLEG): container finished" podID="ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0" containerID="eb8ea9ac6702ca97cde905516c302c3dc78d139cdcf54f1ff2955e33ed2feb5c" exitCode=2 Sep 15 06:41:50.576211 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:50.576066 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" event={"ID":"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0","Type":"ContainerDied","Data":"eb8ea9ac6702ca97cde905516c302c3dc78d139cdcf54f1ff2955e33ed2feb5c"} Sep 15 06:41:50.576211 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:50.576109 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c745ff68c-nfmns" event={"ID":"ddc5b0a6-b900-4fbc-9f1e-c4e1fd6be0a0","Type":"ContainerStarted","Data":"f984da8a1e502e71f86808a968efa72306cd399d4d48e3b17f91615b1fd3df20"} Sep 15 06:41:52.585074 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:52.585041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688d95dd96-8bl7t" event={"ID":"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6","Type":"ContainerStarted","Data":"33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63"} Sep 15 06:41:52.604691 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:52.604642 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-688d95dd96-8bl7t" podStartSLOduration=1.874433819 podStartE2EDuration="4.604626568s" totalImagesPullingTime="2.730192749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:52.602808598 +0000 UTC m=+320.677186158" watchObservedRunningTime="2026-09-15 06:41:52.604626568 +0000 UTC m=+320.679004117" Sep 15 06:41:52.623331 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:52.623302 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:52.726208 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:52.726170 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:52.845935 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:52.845859 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:52.940850 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:52.940818 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:54.590805 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:54.590777 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:54.591183 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:54.590827 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:41:54.610858 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:54.610835 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:54.665520 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:54.665498 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:56.282442 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:56.282413 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:56.335116 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:56.335084 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:58.919830 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:58.919795 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:58.920206 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:58.919857 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:58.924267 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:58.924246 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:41:59.314207 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:59.314179 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:41:59.619031 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:41:59.618956 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:42:01.054548 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:01.054521 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:01.106056 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:01.106027 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:14.321330 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:14.321289 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:14.383638 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:14.383610 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:14.596712 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:14.596640 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:42:14.600307 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:14.600288 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-559d484c58-c2mgc" Sep 15 06:42:17.529886 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:17.529859 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v5vjb_1c787e62-1891-409b-890d-90dc4095665b/serve-healthcheck-canary/0.log" Sep 15 06:42:20.985417 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:20.985389 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:21.085500 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:21.085472 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:21.140813 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:21.140786 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:41.767346 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:41.767256 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:41.842355 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:41.842328 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:46.170586 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:46.170544 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:42:46.172979 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:46.172958 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e225fca-64e1-4eb9-9424-c54b0b77cad6-metrics-certs\") pod \"network-metrics-daemon-xm2jl\" (UID: \"9e225fca-64e1-4eb9-9424-c54b0b77cad6\") " pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:42:46.208636 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:46.208610 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5bkjm\"" Sep 15 06:42:46.216406 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:46.216387 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm2jl" Sep 15 06:42:46.484066 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:46.484044 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xm2jl"] Sep 15 06:42:46.486603 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:42:46.486576 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e225fca_64e1_4eb9_9424_c54b0b77cad6.slice/crio-40f143d3a57e108ae49e826df58d362200c16a5e8cf83dce82a37300de067649 WatchSource:0}: Error finding container 40f143d3a57e108ae49e826df58d362200c16a5e8cf83dce82a37300de067649: Status 404 returned error can't find the container with id 40f143d3a57e108ae49e826df58d362200c16a5e8cf83dce82a37300de067649 Sep 15 06:42:46.788323 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:46.788291 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm2jl" event={"ID":"9e225fca-64e1-4eb9-9424-c54b0b77cad6","Type":"ContainerStarted","Data":"40f143d3a57e108ae49e826df58d362200c16a5e8cf83dce82a37300de067649"} Sep 15 06:42:48.798062 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:48.798021 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm2jl" event={"ID":"9e225fca-64e1-4eb9-9424-c54b0b77cad6","Type":"ContainerStarted","Data":"a22a76989619a0beaa2e047f0229197c30f1920a2fbeb1b6e1b396450ff12a33"} Sep 15 06:42:48.798441 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:48.798088 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm2jl" event={"ID":"9e225fca-64e1-4eb9-9424-c54b0b77cad6","Type":"ContainerStarted","Data":"b51626d63b75b70cf3433a2c9d85348c18f972dbd9786b4064f5474992021e19"} Sep 15 06:42:48.812664 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:48.812619 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xm2jl" podStartSLOduration=375.535563108 podStartE2EDuration="6m16.812601427s" totalImagesPullingTime="1.277038319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:42:48.812108407 +0000 UTC m=+376.886485957" watchObservedRunningTime="2026-09-15 06:42:48.812601427 +0000 UTC m=+376.886978954" Sep 15 06:42:48.896191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:48.896166 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:48.955560 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:48.955537 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:49.170340 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:49.170246 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:49.442890 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:49.442801 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:50.228560 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.228524 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:42:50.228956 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.228779 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="alertmanager" containerID="cri-o://bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d" gracePeriod=120 Sep 15 06:42:50.228956 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.228855 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy-metric" containerID="cri-o://afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008" gracePeriod=120 Sep 15 06:42:50.228956 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.228880 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="prom-label-proxy" containerID="cri-o://54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042" gracePeriod=120 Sep 15 06:42:50.228956 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.228905 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy-web" containerID="cri-o://258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da" gracePeriod=120 Sep 15 06:42:50.229194 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.228932 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="config-reloader" containerID="cri-o://ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be" gracePeriod=120 Sep 15 06:42:50.229194 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.228952 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy" containerID="cri-o://2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5" gracePeriod=120 Sep 15 06:42:50.809061 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.809032 2581 generic.go:356] "Generic (PLEG): container finished" podID="238018be-52e8-46b5-96a4-4234958c24b9" containerID="54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042" exitCode=0 Sep 15 06:42:50.809061 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.809055 2581 generic.go:356] "Generic (PLEG): container finished" podID="238018be-52e8-46b5-96a4-4234958c24b9" containerID="2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5" exitCode=0 Sep 15 06:42:50.809061 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.809062 2581 generic.go:356] "Generic (PLEG): container finished" podID="238018be-52e8-46b5-96a4-4234958c24b9" containerID="ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be" exitCode=0 Sep 15 06:42:50.809061 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.809067 2581 generic.go:356] "Generic (PLEG): container finished" podID="238018be-52e8-46b5-96a4-4234958c24b9" containerID="bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d" exitCode=0 Sep 15 06:42:50.809326 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.809113 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerDied","Data":"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042"} Sep 15 06:42:50.809326 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.809195 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerDied","Data":"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5"} Sep 15 06:42:50.809326 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.809224 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerDied","Data":"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be"} Sep 15 06:42:50.809326 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:50.809247 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerDied","Data":"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d"} Sep 15 06:42:51.481691 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.481670 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.614407 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614314 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-cluster-tls-config\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614407 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614367 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614407 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614390 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-tls-assets\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614407 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614407 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-metrics-client-ca\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614427 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-trusted-ca-bundle\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614461 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614482 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-main-db\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614501 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614530 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-web-config\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614564 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-config-out\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614591 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-web\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614617 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-config-volume\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.614721 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614664 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7tnj\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-kube-api-access-d7tnj\") pod \"238018be-52e8-46b5-96a4-4234958c24b9\" (UID: \"238018be-52e8-46b5-96a4-4234958c24b9\") " Sep 15 06:42:51.615345 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614736 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:42:51.615345 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614826 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 06:42:51.615345 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614944 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-main-db\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.615345 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.614963 2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-metrics-client-ca\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.615532 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.615495 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:42:51.617977 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.617941 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:42:51.618240 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.618213 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:51.619727 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.618910 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-config-volume" (OuterVolumeSpecName: "config-volume") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:51.621425 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.620319 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-kube-api-access-d7tnj" (OuterVolumeSpecName: "kube-api-access-d7tnj") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "kube-api-access-d7tnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:42:51.621425 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.620505 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:51.624888 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.624852 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-config-out" (OuterVolumeSpecName: "config-out") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 06:42:51.624985 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.624941 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:51.626420 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.626389 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:51.627619 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.627589 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:51.634137 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.634119 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-web-config" (OuterVolumeSpecName: "web-config") pod "238018be-52e8-46b5-96a4-4234958c24b9" (UID: "238018be-52e8-46b5-96a4-4234958c24b9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:51.716184 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716138 2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-cluster-tls-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716184 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716182 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716193 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-tls-assets\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716201 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238018be-52e8-46b5-96a4-4234958c24b9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716219 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716228 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-main-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716256 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-web-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716264 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/238018be-52e8-46b5-96a4-4234958c24b9-config-out\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716271 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716280 2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/238018be-52e8-46b5-96a4-4234958c24b9-config-volume\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.716318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.716287 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7tnj\" (UniqueName: \"kubernetes.io/projected/238018be-52e8-46b5-96a4-4234958c24b9-kube-api-access-d7tnj\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:42:51.816101 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.816070 2581 generic.go:356] "Generic (PLEG): container finished" podID="238018be-52e8-46b5-96a4-4234958c24b9" containerID="afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008" exitCode=0 Sep 15 06:42:51.816101 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.816097 2581 generic.go:356] "Generic (PLEG): container finished" podID="238018be-52e8-46b5-96a4-4234958c24b9" containerID="258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da" exitCode=0 Sep 15 06:42:51.816265 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.816125 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerDied","Data":"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008"} Sep 15 06:42:51.816265 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.816167 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.816265 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.816221 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerDied","Data":"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da"} Sep 15 06:42:51.816265 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.816249 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"238018be-52e8-46b5-96a4-4234958c24b9","Type":"ContainerDied","Data":"2b6fb96a82b2b4a1828be02f9dd3234b59ee41fbc57597a3c7f87f646bdb8ec9"} Sep 15 06:42:51.816384 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.816278 2581 scope.go:117] "RemoveContainer" containerID="54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042" Sep 15 06:42:51.824198 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.824179 2581 scope.go:117] "RemoveContainer" containerID="afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008" Sep 15 06:42:51.830963 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.830945 2581 scope.go:117] "RemoveContainer" containerID="2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5" Sep 15 06:42:51.837903 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.837880 2581 scope.go:117] "RemoveContainer" containerID="258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da" Sep 15 06:42:51.838588 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.838568 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:42:51.843842 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.843822 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:42:51.845060 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.845037 2581 scope.go:117] "RemoveContainer" containerID="ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be" Sep 15 06:42:51.854323 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.854302 2581 scope.go:117] "RemoveContainer" containerID="bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d" Sep 15 06:42:51.861012 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.860992 2581 scope.go:117] "RemoveContainer" containerID="fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601" Sep 15 06:42:51.867445 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.867422 2581 scope.go:117] "RemoveContainer" containerID="54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042" Sep 15 06:42:51.868361 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:42:51.868198 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042\": container with ID starting with 54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042 not found: ID does not exist" containerID="54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042" Sep 15 06:42:51.868361 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.868268 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042"} err="failed to get container status \"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042\": rpc error: code = NotFound desc = could not find container \"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042\": container with ID starting with 54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042 not found: ID does not exist" Sep 15 06:42:51.868361 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.868292 2581 scope.go:117] "RemoveContainer" containerID="afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008" Sep 15 06:42:51.868657 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:42:51.868632 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008\": container with ID starting with afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008 not found: ID does not exist" containerID="afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008" Sep 15 06:42:51.868728 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.868662 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008"} err="failed to get container status \"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008\": rpc error: code = NotFound desc = could not find container \"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008\": container with ID starting with afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008 not found: ID does not exist" Sep 15 06:42:51.868728 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.868684 2581 scope.go:117] "RemoveContainer" containerID="2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5" Sep 15 06:42:51.868964 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:42:51.868948 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5\": container with ID starting with 2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5 not found: ID does not exist" containerID="2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5" Sep 15 06:42:51.869031 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.868971 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5"} err="failed to get container status \"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5\": rpc error: code = NotFound desc = could not find container \"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5\": container with ID starting with 2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5 not found: ID does not exist" Sep 15 06:42:51.869031 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.868990 2581 scope.go:117] "RemoveContainer" containerID="258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da" Sep 15 06:42:51.869361 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:42:51.869333 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da\": container with ID starting with 258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da not found: ID does not exist" containerID="258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da" Sep 15 06:42:51.869402 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.869365 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da"} err="failed to get container status \"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da\": rpc error: code = NotFound desc = could not find container \"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da\": container with ID starting with 258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da not found: ID does not exist" Sep 15 06:42:51.869402 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.869379 2581 scope.go:117] "RemoveContainer" containerID="ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be" Sep 15 06:42:51.869656 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:42:51.869635 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be\": container with ID starting with ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be not found: ID does not exist" containerID="ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be" Sep 15 06:42:51.869707 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.869673 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be"} err="failed to get container status \"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be\": rpc error: code = NotFound desc = could not find container \"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be\": container with ID starting with ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be not found: ID does not exist" Sep 15 06:42:51.869707 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.869693 2581 scope.go:117] "RemoveContainer" containerID="bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d" Sep 15 06:42:51.869778 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.869750 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:42:51.869951 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:42:51.869936 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d\": container with ID starting with bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d not found: ID does not exist" containerID="bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d" Sep 15 06:42:51.869999 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.869954 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d"} err="failed to get container status \"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d\": rpc error: code = NotFound desc = could not find container \"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d\": container with ID starting with bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d not found: ID does not exist" Sep 15 06:42:51.869999 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.869967 2581 scope.go:117] "RemoveContainer" containerID="fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601" Sep 15 06:42:51.870070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870005 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy-web" Sep 15 06:42:51.870070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870017 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy-web" Sep 15 06:42:51.870070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870028 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy-metric" Sep 15 06:42:51.870070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870035 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy-metric" Sep 15 06:42:51.870070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870052 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="init-config-reloader" Sep 15 06:42:51.870070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870061 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="init-config-reloader" Sep 15 06:42:51.870070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870070 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="alertmanager" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870079 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="alertmanager" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870091 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="config-reloader" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870099 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="config-reloader" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870108 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870113 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870123 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="prom-label-proxy" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870131 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="prom-label-proxy" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:42:51.870184 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601\": container with ID starting with fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601 not found: ID does not exist" containerID="fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870204 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy-web" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870205 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601"} err="failed to get container status \"fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601\": rpc error: code = NotFound desc = could not find container \"fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601\": container with ID starting with fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601 not found: ID does not exist" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870222 2581 scope.go:117] "RemoveContainer" containerID="54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870215 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870271 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="config-reloader" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870279 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="alertmanager" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870285 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="kube-rbac-proxy-metric" Sep 15 06:42:51.870292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870291 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="238018be-52e8-46b5-96a4-4234958c24b9" containerName="prom-label-proxy" Sep 15 06:42:51.870816 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870466 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042"} err="failed to get container status \"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042\": rpc error: code = NotFound desc = could not find container \"54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042\": container with ID starting with 54555af4ecf34ba7e2696d6ffc3e171376fd36b970da8876bc09905c180cc042 not found: ID does not exist" Sep 15 06:42:51.870816 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870481 2581 scope.go:117] "RemoveContainer" containerID="afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008" Sep 15 06:42:51.870816 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870712 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008"} err="failed to get container status \"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008\": rpc error: code = NotFound desc = could not find container \"afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008\": container with ID starting with afc2e475f16263fd1701af478ab83235089caac4d55ce00839e90fbc4ea8b008 not found: ID does not exist" Sep 15 06:42:51.870816 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870731 2581 scope.go:117] "RemoveContainer" containerID="2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5" Sep 15 06:42:51.870974 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870945 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5"} err="failed to get container status \"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5\": rpc error: code = NotFound desc = could not find container \"2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5\": container with ID starting with 2626dcc068e158415430d294c900f377dfbaed0aa9f318873d6d7d83b33f09d5 not found: ID does not exist" Sep 15 06:42:51.870974 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.870962 2581 scope.go:117] "RemoveContainer" containerID="258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da" Sep 15 06:42:51.871206 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.871190 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da"} err="failed to get container status \"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da\": rpc error: code = NotFound desc = could not find container \"258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da\": container with ID starting with 258a56cd36229516a8c93042271650d74b3dc325495f335249f4d228a59c93da not found: ID does not exist" Sep 15 06:42:51.871258 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.871206 2581 scope.go:117] "RemoveContainer" containerID="ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be" Sep 15 06:42:51.871434 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.871420 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be"} err="failed to get container status \"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be\": rpc error: code = NotFound desc = could not find container \"ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be\": container with ID starting with ed3f2d4f36e71439ede56b6419fca3cc7ae84854297cf7b9c06b6b846b4bf6be not found: ID does not exist" Sep 15 06:42:51.871488 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.871435 2581 scope.go:117] "RemoveContainer" containerID="bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d" Sep 15 06:42:51.871654 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.871639 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d"} err="failed to get container status \"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d\": rpc error: code = NotFound desc = could not find container \"bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d\": container with ID starting with bcd9238d4506280586ea17e32b99b15cb62daa609f96dadaf79ae6adf79b6a2d not found: ID does not exist" Sep 15 06:42:51.871697 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.871654 2581 scope.go:117] "RemoveContainer" containerID="fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601" Sep 15 06:42:51.871851 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.871836 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601"} err="failed to get container status \"fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601\": rpc error: code = NotFound desc = could not find container \"fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601\": container with ID starting with fa13e4d02385b1ceed82487bdf6db6cd2c0e444bac0b7a5cd2c72d3f90806601 not found: ID does not exist" Sep 15 06:42:51.875372 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.875359 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.877990 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.878002 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.878042 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.878051 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.878058 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2bf5d\"" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.878046 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.878089 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.877990 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 15 06:42:51.878203 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.878059 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 15 06:42:51.883306 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.883287 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 15 06:42:51.885685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.885667 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:42:51.917423 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917521 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917521 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917464 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917521 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917490 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917521 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2428de1-c6e9-46f3-8f04-970da01d88d2-config-out\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917641 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917641 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917563 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2428de1-c6e9-46f3-8f04-970da01d88d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917641 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917614 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2428de1-c6e9-46f3-8f04-970da01d88d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917641 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917632 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-web-config\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917758 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917646 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2428de1-c6e9-46f3-8f04-970da01d88d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917758 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917665 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2428de1-c6e9-46f3-8f04-970da01d88d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917758 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917687 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:51.917758 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:51.917706 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vg7m\" (UniqueName: \"kubernetes.io/projected/a2428de1-c6e9-46f3-8f04-970da01d88d2-kube-api-access-7vg7m\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.018827 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.018807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.018916 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.018840 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.018916 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.018861 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.018916 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.018878 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.018916 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.018894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2428de1-c6e9-46f3-8f04-970da01d88d2-config-out\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.018916 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.018911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.019126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.018929 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2428de1-c6e9-46f3-8f04-970da01d88d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.020107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.019273 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2428de1-c6e9-46f3-8f04-970da01d88d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.020107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.019314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-web-config\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.020107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.019342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2428de1-c6e9-46f3-8f04-970da01d88d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.020107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.019357 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2428de1-c6e9-46f3-8f04-970da01d88d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.020107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.019379 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2428de1-c6e9-46f3-8f04-970da01d88d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.020107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.019407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.020107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.019429 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vg7m\" (UniqueName: \"kubernetes.io/projected/a2428de1-c6e9-46f3-8f04-970da01d88d2-kube-api-access-7vg7m\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.021632 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.020733 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2428de1-c6e9-46f3-8f04-970da01d88d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.021632 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.021346 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2428de1-c6e9-46f3-8f04-970da01d88d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.021889 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.021862 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2428de1-c6e9-46f3-8f04-970da01d88d2-config-out\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.022033 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.022007 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.022102 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.022092 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.022174 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.022138 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-web-config\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.022259 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.022240 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.022352 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.022309 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.022398 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.022304 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.023789 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.023774 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2428de1-c6e9-46f3-8f04-970da01d88d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.023892 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.023876 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2428de1-c6e9-46f3-8f04-970da01d88d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.027468 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.027446 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vg7m\" (UniqueName: \"kubernetes.io/projected/a2428de1-c6e9-46f3-8f04-970da01d88d2-kube-api-access-7vg7m\") pod \"alertmanager-main-0\" (UID: \"a2428de1-c6e9-46f3-8f04-970da01d88d2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.185924 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.185859 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 06:42:52.410479 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.410449 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="238018be-52e8-46b5-96a4-4234958c24b9" path="/var/lib/kubelet/pods/238018be-52e8-46b5-96a4-4234958c24b9/volumes" Sep 15 06:42:52.460619 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.460592 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 06:42:52.461632 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:42:52.461601 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2428de1_c6e9_46f3_8f04_970da01d88d2.slice/crio-049f40e90505018d703e901fa9ea9046700c2efd59c43dad70b53f531a4f2f05 WatchSource:0}: Error finding container 049f40e90505018d703e901fa9ea9046700c2efd59c43dad70b53f531a4f2f05: Status 404 returned error can't find the container with id 049f40e90505018d703e901fa9ea9046700c2efd59c43dad70b53f531a4f2f05 Sep 15 06:42:52.821228 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.821198 2581 generic.go:356] "Generic (PLEG): container finished" podID="a2428de1-c6e9-46f3-8f04-970da01d88d2" containerID="cb7994c461af8f1d6baf2a785f2dcb7e26d5707c58ec5ea50c76af8afd56efcc" exitCode=0 Sep 15 06:42:52.821674 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.821297 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2428de1-c6e9-46f3-8f04-970da01d88d2","Type":"ContainerDied","Data":"cb7994c461af8f1d6baf2a785f2dcb7e26d5707c58ec5ea50c76af8afd56efcc"} Sep 15 06:42:52.821674 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:52.821387 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2428de1-c6e9-46f3-8f04-970da01d88d2","Type":"ContainerStarted","Data":"049f40e90505018d703e901fa9ea9046700c2efd59c43dad70b53f531a4f2f05"} Sep 15 06:42:53.247277 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.247242 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:53.347785 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.347710 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:53.829126 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.829095 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2428de1-c6e9-46f3-8f04-970da01d88d2","Type":"ContainerStarted","Data":"63bc850346a7ccdef505e09e83e1c27cd466d71da45eec19a6667839e581d6ea"} Sep 15 06:42:53.829491 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.829199 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2428de1-c6e9-46f3-8f04-970da01d88d2","Type":"ContainerStarted","Data":"37e9ac68cbaebfdc97b7bb71de716f881fa6640206988019b2dfeae0bbf91388"} Sep 15 06:42:53.829491 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.829227 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2428de1-c6e9-46f3-8f04-970da01d88d2","Type":"ContainerStarted","Data":"dd8bb7699926c150106ee3d629edb108a71dfde65fbda4711e4ac7cc7f78d65f"} Sep 15 06:42:53.829491 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.829251 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2428de1-c6e9-46f3-8f04-970da01d88d2","Type":"ContainerStarted","Data":"dc651710395eda64b317dbbb3aeffaf0daaa98c49397c291124035ac27b2e0f5"} Sep 15 06:42:53.829491 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.829275 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2428de1-c6e9-46f3-8f04-970da01d88d2","Type":"ContainerStarted","Data":"045551b4a0b0fba077b433a355013046cf3c72fc904ca91acbfe30c9c15779ef"} Sep 15 06:42:53.829491 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.829298 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2428de1-c6e9-46f3-8f04-970da01d88d2","Type":"ContainerStarted","Data":"e557bef7ebf3da428017a168ae12a70070cb55e60a8d90b3787677243b12fa3e"} Sep 15 06:42:53.854614 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:53.854565 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.839585467 podStartE2EDuration="2.854553064s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.967597ms" isStatefulPod=false podCreationTimestamp="2026-09-15 06:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:42:53.853110224 +0000 UTC m=+381.927487772" watchObservedRunningTime="2026-09-15 06:42:53.854553064 +0000 UTC m=+381.928930612" Sep 15 06:42:54.247689 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.247660 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-548cbcfcc8-gj44b"] Sep 15 06:42:54.250785 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.250768 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.253274 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.253253 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 15 06:42:54.253382 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.253298 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-czwqb\"" Sep 15 06:42:54.253382 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.253298 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 15 06:42:54.253487 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.253400 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 15 06:42:54.253487 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.253470 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 15 06:42:54.253608 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.253593 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 15 06:42:54.258535 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.258514 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 15 06:42:54.264052 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.264031 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-548cbcfcc8-gj44b"] Sep 15 06:42:54.337607 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.337581 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5wft\" (UniqueName: \"kubernetes.io/projected/bbbf190f-82bd-41f4-8489-a8c86093f857-kube-api-access-x5wft\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.337726 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.337612 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.337726 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.337630 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-metrics-client-ca\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.337726 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.337707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-telemeter-trusted-ca-bundle\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.337859 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.337741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-telemeter-client-tls\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.337859 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.337761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-secret-telemeter-client\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.337859 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.337785 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-serving-certs-ca-bundle\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.337984 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.337884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-federate-client-tls\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.439121 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.439098 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wft\" (UniqueName: \"kubernetes.io/projected/bbbf190f-82bd-41f4-8489-a8c86093f857-kube-api-access-x5wft\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.439250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.439129 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.439250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.439162 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-metrics-client-ca\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.439250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.439199 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-telemeter-trusted-ca-bundle\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.439250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.439214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-telemeter-client-tls\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.439250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.439230 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-secret-telemeter-client\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.439250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.439250 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-serving-certs-ca-bundle\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.439528 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.439275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-federate-client-tls\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.440081 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.440051 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-serving-certs-ca-bundle\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.440218 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.440163 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-telemeter-trusted-ca-bundle\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.440317 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.440292 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbbf190f-82bd-41f4-8489-a8c86093f857-metrics-client-ca\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.441957 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.441929 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-federate-client-tls\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.442099 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.442080 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-secret-telemeter-client\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.442160 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.442103 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.442536 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.442518 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bbbf190f-82bd-41f4-8489-a8c86093f857-telemeter-client-tls\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.446995 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.446965 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5wft\" (UniqueName: \"kubernetes.io/projected/bbbf190f-82bd-41f4-8489-a8c86093f857-kube-api-access-x5wft\") pod \"telemeter-client-548cbcfcc8-gj44b\" (UID: \"bbbf190f-82bd-41f4-8489-a8c86093f857\") " pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.561481 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.561402 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" Sep 15 06:42:54.838998 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:54.838974 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-548cbcfcc8-gj44b"] Sep 15 06:42:54.839725 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:42:54.839700 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbbf190f_82bd_41f4_8489_a8c86093f857.slice/crio-886533d74970932c1121af595dba076a9adc2b9cab649c776176ea3733b1ef7a WatchSource:0}: Error finding container 886533d74970932c1121af595dba076a9adc2b9cab649c776176ea3733b1ef7a: Status 404 returned error can't find the container with id 886533d74970932c1121af595dba076a9adc2b9cab649c776176ea3733b1ef7a Sep 15 06:42:55.837230 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:55.837195 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" event={"ID":"bbbf190f-82bd-41f4-8489-a8c86093f857","Type":"ContainerStarted","Data":"886533d74970932c1121af595dba076a9adc2b9cab649c776176ea3733b1ef7a"} Sep 15 06:42:56.843195 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:56.843119 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" event={"ID":"bbbf190f-82bd-41f4-8489-a8c86093f857","Type":"ContainerStarted","Data":"fe5a97af2f38468ff5336f7ebc5826dc7e3872ea66923dbda06150db07d88d15"} Sep 15 06:42:56.843195 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:56.843188 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" event={"ID":"bbbf190f-82bd-41f4-8489-a8c86093f857","Type":"ContainerStarted","Data":"2a2b9ba20a56320576a9334ae336d90f9982b00b84a115efca99cc141f154e71"} Sep 15 06:42:56.843541 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:56.843222 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" event={"ID":"bbbf190f-82bd-41f4-8489-a8c86093f857","Type":"ContainerStarted","Data":"7e6272c24ee805edddbdc7af5b075af70da0c3109a31b73f23204f0aa7725921"} Sep 15 06:42:56.867264 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:56.867219 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-548cbcfcc8-gj44b" podStartSLOduration=1.115212028 podStartE2EDuration="2.867205149s" totalImagesPullingTime="1.751993121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:42:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:42:56.866627913 +0000 UTC m=+384.941005462" watchObservedRunningTime="2026-09-15 06:42:56.867205149 +0000 UTC m=+384.941582696" Sep 15 06:42:57.015279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:57.015252 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:42:57.118784 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:42:57.118707 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:43:08.256917 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:08.256883 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688d95dd96-8bl7t"] Sep 15 06:43:08.351078 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:08.351050 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:43:08.405308 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:08.405276 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:43:33.279985 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.279946 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-688d95dd96-8bl7t" podUID="842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" containerName="console" containerID="cri-o://33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63" gracePeriod=15 Sep 15 06:43:33.522393 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.522369 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:43:33.549246 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549181 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-oauth-config\") pod \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " Sep 15 06:43:33.549358 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549244 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-oauth-serving-cert\") pod \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " Sep 15 06:43:33.549358 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549294 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-serving-cert\") pod \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " Sep 15 06:43:33.549358 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549325 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htnkp\" (UniqueName: \"kubernetes.io/projected/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-kube-api-access-htnkp\") pod \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " Sep 15 06:43:33.549558 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549355 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-trusted-ca-bundle\") pod \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " Sep 15 06:43:33.549558 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549382 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-config\") pod \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " Sep 15 06:43:33.549558 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549459 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-service-ca\") pod \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\" (UID: \"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6\") " Sep 15 06:43:33.549796 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549676 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" (UID: "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:43:33.550134 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549991 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-service-ca" (OuterVolumeSpecName: "service-ca") pod "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" (UID: "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:43:33.550134 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.550008 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" (UID: "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:43:33.550134 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.549973 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-config" (OuterVolumeSpecName: "console-config") pod "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" (UID: "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:43:33.551546 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.551521 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" (UID: "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:43:33.551724 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.551702 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" (UID: "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:43:33.551785 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.551704 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-kube-api-access-htnkp" (OuterVolumeSpecName: "kube-api-access-htnkp") pod "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" (UID: "842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6"). InnerVolumeSpecName "kube-api-access-htnkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:43:33.650519 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.650466 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-service-ca\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:43:33.650519 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.650511 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-oauth-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:43:33.650519 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.650521 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-oauth-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:43:33.650519 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.650529 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:43:33.650519 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.650536 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htnkp\" (UniqueName: \"kubernetes.io/projected/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-kube-api-access-htnkp\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:43:33.650802 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.650544 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-trusted-ca-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:43:33.650802 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.650553 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6-console-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:43:33.984407 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.984326 2581 generic.go:356] "Generic (PLEG): container finished" podID="842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" containerID="33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63" exitCode=0 Sep 15 06:43:33.984407 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.984368 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688d95dd96-8bl7t" Sep 15 06:43:33.984407 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.984376 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688d95dd96-8bl7t" event={"ID":"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6","Type":"ContainerDied","Data":"33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63"} Sep 15 06:43:33.984649 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.984421 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688d95dd96-8bl7t" event={"ID":"842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6","Type":"ContainerDied","Data":"74c7a6334897f94099e31dcb3bfb30719a7b4a8a4572ad29487a35d77221d7c0"} Sep 15 06:43:33.984649 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.984439 2581 scope.go:117] "RemoveContainer" containerID="33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63" Sep 15 06:43:33.992698 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.992681 2581 scope.go:117] "RemoveContainer" containerID="33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63" Sep 15 06:43:33.992947 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:43:33.992930 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63\": container with ID starting with 33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63 not found: ID does not exist" containerID="33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63" Sep 15 06:43:33.992984 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:33.992955 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63"} err="failed to get container status \"33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63\": rpc error: code = NotFound desc = could not find container \"33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63\": container with ID starting with 33e6de7bd145968d42c1be3e71f5c639abb0fabb6ece73e45b4f63c896829d63 not found: ID does not exist" Sep 15 06:43:34.004382 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:34.004359 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688d95dd96-8bl7t"] Sep 15 06:43:34.009730 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:34.009709 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-688d95dd96-8bl7t"] Sep 15 06:43:34.410502 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:34.410470 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" path="/var/lib/kubelet/pods/842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6/volumes" Sep 15 06:43:47.694499 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:47.694465 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:43:47.744157 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:43:47.744117 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:04.402956 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.402919 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:04.471972 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.471940 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b6d8489c8-qtx97"] Sep 15 06:44:04.472298 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.472283 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" containerName="console" Sep 15 06:44:04.472351 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.472300 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" containerName="console" Sep 15 06:44:04.472385 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.472365 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="842fbd95-0a52-42eb-9ae4-7ac9e8a64fb6" containerName="console" Sep 15 06:44:04.475241 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.475220 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.475942 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.475917 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:04.477690 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.477674 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 15 06:44:04.477875 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.477856 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 15 06:44:04.477993 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.477884 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 15 06:44:04.478693 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.478676 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 15 06:44:04.478838 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.478818 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 15 06:44:04.478939 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.478844 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 15 06:44:04.478939 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.478869 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ksjg5\"" Sep 15 06:44:04.478939 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.478818 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 15 06:44:04.486751 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.486719 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 15 06:44:04.488431 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.487579 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b6d8489c8-qtx97"] Sep 15 06:44:04.583023 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.582986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-serving-cert\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.583224 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.583058 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-oauth-serving-cert\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.583224 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.583090 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-trusted-ca-bundle\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.583224 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.583103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l5kj\" (UniqueName: \"kubernetes.io/projected/8b59513e-9d36-42e1-8885-4c73e4791f15-kube-api-access-4l5kj\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.583224 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.583138 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-oauth-config\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.583377 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.583235 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-console-config\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.583377 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.583259 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-service-ca\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.598922 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.598878 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:04.656593 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.656507 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:04.683668 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.683640 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-oauth-serving-cert\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.683744 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.683681 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-trusted-ca-bundle\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.683744 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.683701 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l5kj\" (UniqueName: \"kubernetes.io/projected/8b59513e-9d36-42e1-8885-4c73e4791f15-kube-api-access-4l5kj\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.683822 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.683741 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-oauth-config\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.683822 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.683776 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-console-config\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.683822 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.683798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-service-ca\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.683953 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.683846 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-serving-cert\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.684480 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.684457 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-oauth-serving-cert\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.684607 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.684582 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-console-config\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.684653 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.684592 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-service-ca\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.684818 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.684798 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-trusted-ca-bundle\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.686405 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.686385 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-oauth-config\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.686491 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.686391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-serving-cert\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.691319 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.691298 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l5kj\" (UniqueName: \"kubernetes.io/projected/8b59513e-9d36-42e1-8885-4c73e4791f15-kube-api-access-4l5kj\") pod \"console-b6d8489c8-qtx97\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.787786 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.787751 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:04.876048 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.876017 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:04.928619 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:04.928551 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:05.055774 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:05.055752 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b6d8489c8-qtx97"] Sep 15 06:44:05.058374 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:44:05.058345 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b59513e_9d36_42e1_8885_4c73e4791f15.slice/crio-f8789f68ee5acfcd8fe290237528875bad18dd055412bad3fce12617bf6bd59d WatchSource:0}: Error finding container f8789f68ee5acfcd8fe290237528875bad18dd055412bad3fce12617bf6bd59d: Status 404 returned error can't find the container with id f8789f68ee5acfcd8fe290237528875bad18dd055412bad3fce12617bf6bd59d Sep 15 06:44:05.104959 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:05.104936 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6d8489c8-qtx97" event={"ID":"8b59513e-9d36-42e1-8885-4c73e4791f15","Type":"ContainerStarted","Data":"f8789f68ee5acfcd8fe290237528875bad18dd055412bad3fce12617bf6bd59d"} Sep 15 06:44:05.671590 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:05.671544 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:05.719924 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:05.719899 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:06.110841 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:06.110807 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6d8489c8-qtx97" event={"ID":"8b59513e-9d36-42e1-8885-4c73e4791f15","Type":"ContainerStarted","Data":"ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93"} Sep 15 06:44:06.128127 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:06.128077 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b6d8489c8-qtx97" podStartSLOduration=2.128056375 podStartE2EDuration="2.128056375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:44:06.125943617 +0000 UTC m=+454.200321166" watchObservedRunningTime="2026-09-15 06:44:06.128056375 +0000 UTC m=+454.202433927" Sep 15 06:44:14.787944 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:14.787907 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:14.788365 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:14.787966 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:14.792349 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:14.792329 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:15.149278 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:15.149205 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:44:15.288675 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:15.288648 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:15.393598 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:15.393569 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:15.442427 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:15.442364 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:16.342296 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:16.342267 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:16.396747 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:16.396715 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:29.328716 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:29.328686 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:29.430636 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:29.430606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:44:34.354116 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:44:34.354079 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:33.297895 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:33.297857 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:33.386994 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:33.386954 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:35.246281 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:35.246238 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:36.771862 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:36.771823 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:46.493325 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:46.493288 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:46.554435 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:46.554404 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:47.196704 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:47.196670 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:48.721477 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.721437 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bff9b8dcb-v2c9w"] Sep 15 06:46:48.724553 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.724538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.735889 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.735868 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bff9b8dcb-v2c9w"] Sep 15 06:46:48.766538 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.766512 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:46:48.789700 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.789676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-oauth-config\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.789791 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.789712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-trusted-ca-bundle\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.789791 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.789731 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-service-ca\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.789874 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.789797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-serving-cert\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.789874 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.789822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-config\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.789874 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.789858 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-oauth-serving-cert\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.789976 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.789874 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l8xq\" (UniqueName: \"kubernetes.io/projected/e9fa5342-06b9-4d5f-8761-3335c50720d7-kube-api-access-4l8xq\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.890579 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.890549 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-service-ca\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.890685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.890594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-serving-cert\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.890685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.890620 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-config\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.890685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.890656 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-oauth-serving-cert\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.890685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.890680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l8xq\" (UniqueName: \"kubernetes.io/projected/e9fa5342-06b9-4d5f-8761-3335c50720d7-kube-api-access-4l8xq\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.890869 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.890732 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-oauth-config\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.890869 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.890773 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-trusted-ca-bundle\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.891444 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.891418 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-oauth-serving-cert\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.891444 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.891431 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-service-ca\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.891592 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.891566 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-trusted-ca-bundle\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.891649 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.891629 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-config\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.893580 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.893539 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-oauth-config\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.893864 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.893844 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9fa5342-06b9-4d5f-8761-3335c50720d7-console-serving-cert\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:48.901727 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:48.901709 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l8xq\" (UniqueName: \"kubernetes.io/projected/e9fa5342-06b9-4d5f-8761-3335c50720d7-kube-api-access-4l8xq\") pod \"console-5bff9b8dcb-v2c9w\" (UID: \"e9fa5342-06b9-4d5f-8761-3335c50720d7\") " pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:49.033765 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:49.033725 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:49.302356 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:49.302282 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bff9b8dcb-v2c9w"] Sep 15 06:46:49.305181 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:46:49.305134 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fa5342_06b9_4d5f_8761_3335c50720d7.slice/crio-ddc10855d0ef87889608099b81d40cddd85dbc59fcabd3f2471f6953fd80a06a WatchSource:0}: Error finding container ddc10855d0ef87889608099b81d40cddd85dbc59fcabd3f2471f6953fd80a06a: Status 404 returned error can't find the container with id ddc10855d0ef87889608099b81d40cddd85dbc59fcabd3f2471f6953fd80a06a Sep 15 06:46:49.749475 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:49.749451 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bff9b8dcb-v2c9w" event={"ID":"e9fa5342-06b9-4d5f-8761-3335c50720d7","Type":"ContainerStarted","Data":"b1bc3ecb40d91303f69ecb397eaa1674dcb2cd5b9c4da7f30724904749c55848"} Sep 15 06:46:49.749840 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:49.749500 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bff9b8dcb-v2c9w" event={"ID":"e9fa5342-06b9-4d5f-8761-3335c50720d7","Type":"ContainerStarted","Data":"ddc10855d0ef87889608099b81d40cddd85dbc59fcabd3f2471f6953fd80a06a"} Sep 15 06:46:49.766405 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:49.766363 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bff9b8dcb-v2c9w" podStartSLOduration=1.7663487820000001 podStartE2EDuration="1.766348782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:46:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:46:49.765193282 +0000 UTC m=+617.839570832" watchObservedRunningTime="2026-09-15 06:46:49.766348782 +0000 UTC m=+617.840726333" Sep 15 06:46:59.033988 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:59.033887 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:59.033988 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:59.033962 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:59.038728 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:59.038705 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:59.794220 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:59.794188 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bff9b8dcb-v2c9w" Sep 15 06:46:59.837093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:59.837050 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b6d8489c8-qtx97"] Sep 15 06:46:59.937465 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:46:59.937433 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:00.041878 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:00.041846 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:00.142057 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:00.141982 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:13.047315 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:13.047277 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:16.598857 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:16.598824 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:16.653374 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:16.653309 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:23.100640 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.100599 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:23.311374 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.311342 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:23.690313 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.690284 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:23.793345 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.793310 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:23.795737 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.795703 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l"] Sep 15 06:47:23.800247 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.800220 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:23.802357 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.802338 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 15 06:47:23.802458 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.802344 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 15 06:47:23.802611 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.802597 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 15 06:47:23.802694 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.802676 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 15 06:47:23.802849 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.802835 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-rmd7v\"" Sep 15 06:47:23.802900 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.802877 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 15 06:47:23.802974 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.802962 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 15 06:47:23.803027 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.802991 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 15 06:47:23.810390 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.808873 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l"] Sep 15 06:47:23.971636 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.971532 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:23.992890 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.992858 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:23.993061 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.992905 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82ea2aa6-8164-4a61-8624-50975d678a12-feast-data\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:23.993061 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.992951 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-online-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:23.993061 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.992975 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-ui-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:23.993061 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.993010 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-offline-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:23.993260 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:23.993099 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjm4p\" (UniqueName: \"kubernetes.io/projected/82ea2aa6-8164-4a61-8624-50975d678a12-kube-api-access-pjm4p\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.072324 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.072288 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:24.093668 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.093639 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82ea2aa6-8164-4a61-8624-50975d678a12-feast-data\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.093815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.093672 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-online-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.093815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.093689 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-ui-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.093815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.093711 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-offline-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.093815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.093750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjm4p\" (UniqueName: \"kubernetes.io/projected/82ea2aa6-8164-4a61-8624-50975d678a12-kube-api-access-pjm4p\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.094018 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.093831 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.094018 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:47:24.093911 2581 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 15 06:47:24.094018 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:47:24.093966 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls podName:82ea2aa6-8164-4a61-8624-50975d678a12 nodeName:}" failed. No retries permitted until 2026-09-15 06:47:24.59394806 +0000 UTC m=+652.668325589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls") pod "feast-simple-feast-setup-6b7c59ff95-2r97l" (UID: "82ea2aa6-8164-4a61-8624-50975d678a12") : secret "feast-simple-feast-setup-registry-tls" not found Sep 15 06:47:24.094240 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.094111 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82ea2aa6-8164-4a61-8624-50975d678a12-feast-data\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.096528 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.096504 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-online-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.096629 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.096527 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-offline-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.096629 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.096543 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-ui-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.103967 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.103942 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjm4p\" (UniqueName: \"kubernetes.io/projected/82ea2aa6-8164-4a61-8624-50975d678a12-kube-api-access-pjm4p\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.173401 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.173362 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:24.273970 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.273938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:24.599082 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.598988 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.601590 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.601567 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-2r97l\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.713416 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.713379 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:47:24.857080 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.856972 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b6d8489c8-qtx97" podUID="8b59513e-9d36-42e1-8885-4c73e4791f15" containerName="console" containerID="cri-o://ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93" gracePeriod=15 Sep 15 06:47:24.990815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.990784 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l"] Sep 15 06:47:24.991791 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:47:24.991765 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82ea2aa6_8164_4a61_8624_50975d678a12.slice/crio-f445d50a7ff2b660f27fae70f2552c9d7d897023e47307055c9d7e824c25e7f0 WatchSource:0}: Error finding container f445d50a7ff2b660f27fae70f2552c9d7d897023e47307055c9d7e824c25e7f0: Status 404 returned error can't find the container with id f445d50a7ff2b660f27fae70f2552c9d7d897023e47307055c9d7e824c25e7f0 Sep 15 06:47:24.993793 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:24.993773 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 06:47:25.090912 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.090887 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:47:25.107353 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107291 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-service-ca\") pod \"8b59513e-9d36-42e1-8885-4c73e4791f15\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " Sep 15 06:47:25.107754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107362 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-trusted-ca-bundle\") pod \"8b59513e-9d36-42e1-8885-4c73e4791f15\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " Sep 15 06:47:25.107754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107402 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-serving-cert\") pod \"8b59513e-9d36-42e1-8885-4c73e4791f15\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " Sep 15 06:47:25.107754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107433 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-oauth-serving-cert\") pod \"8b59513e-9d36-42e1-8885-4c73e4791f15\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " Sep 15 06:47:25.107754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107460 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-oauth-config\") pod \"8b59513e-9d36-42e1-8885-4c73e4791f15\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " Sep 15 06:47:25.107754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107501 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l5kj\" (UniqueName: \"kubernetes.io/projected/8b59513e-9d36-42e1-8885-4c73e4791f15-kube-api-access-4l5kj\") pod \"8b59513e-9d36-42e1-8885-4c73e4791f15\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " Sep 15 06:47:25.107754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107533 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-console-config\") pod \"8b59513e-9d36-42e1-8885-4c73e4791f15\" (UID: \"8b59513e-9d36-42e1-8885-4c73e4791f15\") " Sep 15 06:47:25.107754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107649 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-service-ca" (OuterVolumeSpecName: "service-ca") pod "8b59513e-9d36-42e1-8885-4c73e4791f15" (UID: "8b59513e-9d36-42e1-8885-4c73e4791f15"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:47:25.107754 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107746 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-service-ca\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:47:25.108283 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107868 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8b59513e-9d36-42e1-8885-4c73e4791f15" (UID: "8b59513e-9d36-42e1-8885-4c73e4791f15"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:47:25.108283 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.107979 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8b59513e-9d36-42e1-8885-4c73e4791f15" (UID: "8b59513e-9d36-42e1-8885-4c73e4791f15"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:47:25.108529 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.108502 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-console-config" (OuterVolumeSpecName: "console-config") pod "8b59513e-9d36-42e1-8885-4c73e4791f15" (UID: "8b59513e-9d36-42e1-8885-4c73e4791f15"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:47:25.110179 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.110108 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8b59513e-9d36-42e1-8885-4c73e4791f15" (UID: "8b59513e-9d36-42e1-8885-4c73e4791f15"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:47:25.110576 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.110435 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8b59513e-9d36-42e1-8885-4c73e4791f15" (UID: "8b59513e-9d36-42e1-8885-4c73e4791f15"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:47:25.110812 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.110788 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b59513e-9d36-42e1-8885-4c73e4791f15-kube-api-access-4l5kj" (OuterVolumeSpecName: "kube-api-access-4l5kj") pod "8b59513e-9d36-42e1-8885-4c73e4791f15" (UID: "8b59513e-9d36-42e1-8885-4c73e4791f15"). InnerVolumeSpecName "kube-api-access-4l5kj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:47:25.208411 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.208374 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-trusted-ca-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:47:25.208411 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.208404 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:47:25.208411 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.208413 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-oauth-serving-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:47:25.208623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.208421 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b59513e-9d36-42e1-8885-4c73e4791f15-console-oauth-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:47:25.208623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.208430 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l5kj\" (UniqueName: \"kubernetes.io/projected/8b59513e-9d36-42e1-8885-4c73e4791f15-kube-api-access-4l5kj\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:47:25.208623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.208439 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b59513e-9d36-42e1-8885-4c73e4791f15-console-config\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:47:25.894079 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.894038 2581 generic.go:356] "Generic (PLEG): container finished" podID="8b59513e-9d36-42e1-8885-4c73e4791f15" containerID="ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93" exitCode=0 Sep 15 06:47:25.894286 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.894121 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6d8489c8-qtx97" Sep 15 06:47:25.894286 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.894213 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6d8489c8-qtx97" event={"ID":"8b59513e-9d36-42e1-8885-4c73e4791f15","Type":"ContainerDied","Data":"ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93"} Sep 15 06:47:25.894286 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.894280 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6d8489c8-qtx97" event={"ID":"8b59513e-9d36-42e1-8885-4c73e4791f15","Type":"ContainerDied","Data":"f8789f68ee5acfcd8fe290237528875bad18dd055412bad3fce12617bf6bd59d"} Sep 15 06:47:25.894456 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.894308 2581 scope.go:117] "RemoveContainer" containerID="ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93" Sep 15 06:47:25.897350 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.897322 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerStarted","Data":"f445d50a7ff2b660f27fae70f2552c9d7d897023e47307055c9d7e824c25e7f0"} Sep 15 06:47:25.909219 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.909189 2581 scope.go:117] "RemoveContainer" containerID="ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93" Sep 15 06:47:25.909604 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:47:25.909575 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93\": container with ID starting with ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93 not found: ID does not exist" containerID="ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93" Sep 15 06:47:25.909757 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.909617 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93"} err="failed to get container status \"ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93\": rpc error: code = NotFound desc = could not find container \"ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93\": container with ID starting with ab7655b327673e32526654afd2e9ec3964f7b9032d64dbb1168f9ead80d86b93 not found: ID does not exist" Sep 15 06:47:25.921123 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.921094 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b6d8489c8-qtx97"] Sep 15 06:47:25.927571 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:25.927547 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b6d8489c8-qtx97"] Sep 15 06:47:26.412163 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:26.412105 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b59513e-9d36-42e1-8885-4c73e4791f15" path="/var/lib/kubelet/pods/8b59513e-9d36-42e1-8885-4c73e4791f15/volumes" Sep 15 06:47:51.768893 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:51.768863 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:47:52.029463 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:52.029382 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerStarted","Data":"8722a6590873de0a65048c550e33ae0a8ac313857d298b4e74c0e89b9ffebb15"} Sep 15 06:47:57.052406 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:57.052366 2581 generic.go:356] "Generic (PLEG): container finished" podID="82ea2aa6-8164-4a61-8624-50975d678a12" containerID="8722a6590873de0a65048c550e33ae0a8ac313857d298b4e74c0e89b9ffebb15" exitCode=0 Sep 15 06:47:57.052867 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:57.052440 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerDied","Data":"8722a6590873de0a65048c550e33ae0a8ac313857d298b4e74c0e89b9ffebb15"} Sep 15 06:47:58.059040 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:47:58.059005 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerStarted","Data":"87e27ac944046f3dd260509e01803ba1ee249e5ccc1abfc8f9bf472d78a2968f"} Sep 15 06:48:04.086853 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:04.086821 2581 generic.go:356] "Generic (PLEG): container finished" podID="82ea2aa6-8164-4a61-8624-50975d678a12" containerID="87e27ac944046f3dd260509e01803ba1ee249e5ccc1abfc8f9bf472d78a2968f" exitCode=0 Sep 15 06:48:04.087297 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:04.086876 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerDied","Data":"87e27ac944046f3dd260509e01803ba1ee249e5ccc1abfc8f9bf472d78a2968f"} Sep 15 06:48:04.091170 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:04.091119 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 15 06:48:05.103387 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:05.103304 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerStarted","Data":"645cc0f5685dddb87aeac91434d40509daee00fbd754c6d9091ebf6ce3f58895"} Sep 15 06:48:05.103387 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:05.103363 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerStarted","Data":"165373e8f751ba3569fab82c09bace189af333621c8c328a9c39ee5bcbf15c88"} Sep 15 06:48:05.103387 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:05.103382 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerStarted","Data":"ff418f091814049ea1a42dcacb6f15612c39b3e033aa250d77c54294e892ace2"} Sep 15 06:48:05.104050 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:05.103400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerStarted","Data":"e0b3e16321a43e4f652b76c02f7523ad135c73e8ca24ef9930388acdba12f2f7"} Sep 15 06:48:05.128113 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:05.128039 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podStartSLOduration=15.724247895 podStartE2EDuration="42.12801303s" totalImagesPullingTime="26.403765135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:48:05.125616923 +0000 UTC m=+693.199994509" watchObservedRunningTime="2026-09-15 06:48:05.12801303 +0000 UTC m=+693.202390587" Sep 15 06:48:06.713776 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:06.713743 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:06.714321 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:06.714302 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:06.714824 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:06.714801 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:06.714933 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:06.714846 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:06.715789 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:06.715753 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.22:6571: connect: connection refused" Sep 15 06:48:06.726307 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:06.715782 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Sep 15 06:48:06.726512 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:06.715837 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused" Sep 15 06:48:06.726512 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:06.715864 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 15 06:48:09.714826 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:09.714776 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 15 06:48:09.715568 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:09.714814 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Sep 15 06:48:09.715674 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:09.714875 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.22:6571: connect: connection refused" Sep 15 06:48:09.715741 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:09.715130 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused" Sep 15 06:48:12.713967 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:12.713904 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 15 06:48:12.714556 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:12.714534 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:12.715097 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:12.715069 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:12.715238 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:12.715116 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:12.715458 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:12.715414 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:12.724355 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:12.720512 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:13.149722 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:13.149673 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:13.149910 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:13.149735 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:13.150159 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:13.150116 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:13.153049 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:13.153028 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:14.714576 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:14.714537 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:15.714945 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:15.714911 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:48:16.167555 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:48:16.167527 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:49:13.773671 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:13.773636 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:13.839325 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:13.839287 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:18.014340 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.014272 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:18.022901 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.022874 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l"] Sep 15 06:49:18.023134 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.023106 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="registry" containerID="cri-o://e0b3e16321a43e4f652b76c02f7523ad135c73e8ca24ef9930388acdba12f2f7" gracePeriod=30 Sep 15 06:49:18.023372 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.023199 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="offline" containerID="cri-o://165373e8f751ba3569fab82c09bace189af333621c8c328a9c39ee5bcbf15c88" gracePeriod=30 Sep 15 06:49:18.023498 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.023241 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" containerID="cri-o://ff418f091814049ea1a42dcacb6f15612c39b3e033aa250d77c54294e892ace2" gracePeriod=30 Sep 15 06:49:18.023498 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.023257 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" containerID="cri-o://645cc0f5685dddb87aeac91434d40509daee00fbd754c6d9091ebf6ce3f58895" gracePeriod=30 Sep 15 06:49:18.114162 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.114112 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:18.279650 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.278839 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:18.386924 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.386888 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:18.388098 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.388063 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d"] Sep 15 06:49:18.388468 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.388451 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b59513e-9d36-42e1-8885-4c73e4791f15" containerName="console" Sep 15 06:49:18.388468 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.388467 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b59513e-9d36-42e1-8885-4c73e4791f15" containerName="console" Sep 15 06:49:18.388612 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.388559 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b59513e-9d36-42e1-8885-4c73e4791f15" containerName="console" Sep 15 06:49:18.392349 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.392330 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.394826 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.394808 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-h4p6b\"" Sep 15 06:49:18.411853 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.411829 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d"] Sep 15 06:49:18.446039 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.445993 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:18.463364 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.463325 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tpzm\" (UniqueName: \"kubernetes.io/projected/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-kube-api-access-7tpzm\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.463617 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.463443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-online-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.463617 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.463471 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-feast-data\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.463617 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.463572 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.463617 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.463609 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.463823 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.463689 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-registry-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.564438 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.564330 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-registry-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.564438 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.564382 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tpzm\" (UniqueName: \"kubernetes.io/projected/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-kube-api-access-7tpzm\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.564438 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.564441 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-online-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.564700 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.564462 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-feast-data\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.564700 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.564514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.564700 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.564538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.564700 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:49:18.564657 2581 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 15 06:49:18.564878 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:49:18.564727 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls podName:1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde nodeName:}" failed. No retries permitted until 2026-09-15 06:49:19.064706135 +0000 UTC m=+767.139083670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls") pod "feast-simple-feast-setup-6b7c59ff95-wc58d" (UID: "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde") : secret "feast-simple-feast-setup-offline-tls" not found Sep 15 06:49:18.564878 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:49:18.564858 2581 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 15 06:49:18.565004 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:49:18.564890 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls podName:1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde nodeName:}" failed. No retries permitted until 2026-09-15 06:49:19.064879772 +0000 UTC m=+767.139257306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls") pod "feast-simple-feast-setup-6b7c59ff95-wc58d" (UID: "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde") : secret "feast-simple-feast-setup-ui-tls" not found Sep 15 06:49:18.565287 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.565262 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-feast-data\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.567848 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.567779 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-online-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.568013 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.567990 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-registry-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.574619 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.574575 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tpzm\" (UniqueName: \"kubernetes.io/projected/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-kube-api-access-7tpzm\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:18.718162 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.718064 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:18.835367 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:18.835302 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:49:19.069454 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.069423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:19.069788 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.069460 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:19.071903 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.071881 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:19.071998 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.071930 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls\") pod \"feast-simple-feast-setup-6b7c59ff95-wc58d\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:19.302979 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.302952 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:19.420753 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.420726 2581 generic.go:356] "Generic (PLEG): container finished" podID="82ea2aa6-8164-4a61-8624-50975d678a12" containerID="645cc0f5685dddb87aeac91434d40509daee00fbd754c6d9091ebf6ce3f58895" exitCode=0 Sep 15 06:49:19.420753 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.420747 2581 generic.go:356] "Generic (PLEG): container finished" podID="82ea2aa6-8164-4a61-8624-50975d678a12" containerID="ff418f091814049ea1a42dcacb6f15612c39b3e033aa250d77c54294e892ace2" exitCode=0 Sep 15 06:49:19.420930 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.420808 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerDied","Data":"645cc0f5685dddb87aeac91434d40509daee00fbd754c6d9091ebf6ce3f58895"} Sep 15 06:49:19.420930 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.420845 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerDied","Data":"ff418f091814049ea1a42dcacb6f15612c39b3e033aa250d77c54294e892ace2"} Sep 15 06:49:19.577551 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:19.577522 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d"] Sep 15 06:49:19.579205 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:49:19.579181 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e5350cb_8ea0_4987_9fdb_7c1edb0a6bde.slice/crio-7a1f4a35e2a3dfe12885c916b7da8cfa8b92ea2e0e8f9231a54691469788b9a5 WatchSource:0}: Error finding container 7a1f4a35e2a3dfe12885c916b7da8cfa8b92ea2e0e8f9231a54691469788b9a5: Status 404 returned error can't find the container with id 7a1f4a35e2a3dfe12885c916b7da8cfa8b92ea2e0e8f9231a54691469788b9a5 Sep 15 06:49:20.425841 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:20.425802 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerStarted","Data":"add146156052169447243e53a3b2c96e5e1f4464853a0736a3828dbc981e3fd0"} Sep 15 06:49:20.426288 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:20.425856 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerStarted","Data":"7a1f4a35e2a3dfe12885c916b7da8cfa8b92ea2e0e8f9231a54691469788b9a5"} Sep 15 06:49:23.150407 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:23.150365 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Sep 15 06:49:25.448921 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:25.448887 2581 generic.go:356] "Generic (PLEG): container finished" podID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerID="add146156052169447243e53a3b2c96e5e1f4464853a0736a3828dbc981e3fd0" exitCode=0 Sep 15 06:49:25.449293 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:25.448970 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerDied","Data":"add146156052169447243e53a3b2c96e5e1f4464853a0736a3828dbc981e3fd0"} Sep 15 06:49:26.167115 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:26.167061 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 15 06:49:26.454650 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:26.454565 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerStarted","Data":"54b7cf0323617b643d533564b964eef433f61ce6fdf74d8194e861434fc77168"} Sep 15 06:49:31.477706 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:31.477673 2581 generic.go:356] "Generic (PLEG): container finished" podID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerID="54b7cf0323617b643d533564b964eef433f61ce6fdf74d8194e861434fc77168" exitCode=0 Sep 15 06:49:31.478076 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:31.477731 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerDied","Data":"54b7cf0323617b643d533564b964eef433f61ce6fdf74d8194e861434fc77168"} Sep 15 06:49:32.488804 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:32.488758 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerStarted","Data":"e4d9cc0571cfcb6406620258f2a6bddd5fd4ad131fa32834bb2ac09a7e17f74a"} Sep 15 06:49:32.489286 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:32.488830 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerStarted","Data":"73f6869337da9bf9c1b359738cce6e8638c5156fe2743216c87fe0bb3fa3b039"} Sep 15 06:49:32.489286 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:32.488852 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerStarted","Data":"abdedf8110c22809e731ed5dc7498d58bdc9e435875a9d95bf3a7130d4c4588f"} Sep 15 06:49:32.489286 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:32.488870 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerStarted","Data":"9c753546f417e76ec2f4d2d8f45096e37ee52a228039d298f262ca27311e5d31"} Sep 15 06:49:32.527782 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:32.527713 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podStartSLOduration=14.527689253 podStartE2EDuration="14.527689253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:49:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:49:32.524594249 +0000 UTC m=+780.598971800" watchObservedRunningTime="2026-09-15 06:49:32.527689253 +0000 UTC m=+780.602066817" Sep 15 06:49:33.150541 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:33.150493 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Sep 15 06:49:34.303998 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:34.303958 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:34.304515 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:34.304018 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:34.304515 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:34.304036 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:34.304515 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:34.304052 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:34.306370 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:34.306332 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.23:8816: connect: connection refused" Sep 15 06:49:34.306609 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:34.306372 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 15 06:49:34.306710 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:34.306381 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.23:6571: connect: connection refused" Sep 15 06:49:34.306710 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:34.306459 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 15 06:49:36.166787 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:36.166724 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 15 06:49:37.303586 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:37.303542 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 15 06:49:37.304131 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:37.303542 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.23:8816: connect: connection refused" Sep 15 06:49:37.304285 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:37.303585 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 15 06:49:37.304285 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:37.303737 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.23:6571: connect: connection refused" Sep 15 06:49:39.303545 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:39.303493 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:39.304029 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:39.303573 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:39.304029 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:39.303595 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:39.304029 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:39.303615 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:40.304761 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:40.304722 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:40.305212 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:40.305051 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:40.305421 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:40.305400 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:40.305549 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:40.305535 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:40.305609 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:40.305601 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:40.310265 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:40.310247 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:40.538139 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:40.538098 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:40.541425 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:40.541403 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:49:43.149966 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:43.149925 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Sep 15 06:49:43.150450 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:43.150046 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:49:46.166880 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:46.166830 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 15 06:49:46.167418 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:46.166971 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:49:48.573916 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.573890 2581 generic.go:356] "Generic (PLEG): container finished" podID="82ea2aa6-8164-4a61-8624-50975d678a12" containerID="165373e8f751ba3569fab82c09bace189af333621c8c328a9c39ee5bcbf15c88" exitCode=137 Sep 15 06:49:48.573916 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.573911 2581 generic.go:356] "Generic (PLEG): container finished" podID="82ea2aa6-8164-4a61-8624-50975d678a12" containerID="e0b3e16321a43e4f652b76c02f7523ad135c73e8ca24ef9930388acdba12f2f7" exitCode=137 Sep 15 06:49:48.574395 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.573964 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerDied","Data":"165373e8f751ba3569fab82c09bace189af333621c8c328a9c39ee5bcbf15c88"} Sep 15 06:49:48.574395 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.574023 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerDied","Data":"e0b3e16321a43e4f652b76c02f7523ad135c73e8ca24ef9930388acdba12f2f7"} Sep 15 06:49:48.667401 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.667379 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:49:48.746281 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.746253 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-ui-tls\") pod \"82ea2aa6-8164-4a61-8624-50975d678a12\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " Sep 15 06:49:48.746416 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.746313 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls\") pod \"82ea2aa6-8164-4a61-8624-50975d678a12\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " Sep 15 06:49:48.746416 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.746340 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82ea2aa6-8164-4a61-8624-50975d678a12-feast-data\") pod \"82ea2aa6-8164-4a61-8624-50975d678a12\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " Sep 15 06:49:48.746527 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.746433 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-online-tls\") pod \"82ea2aa6-8164-4a61-8624-50975d678a12\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " Sep 15 06:49:48.746527 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.746475 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-offline-tls\") pod \"82ea2aa6-8164-4a61-8624-50975d678a12\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " Sep 15 06:49:48.746527 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.746492 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjm4p\" (UniqueName: \"kubernetes.io/projected/82ea2aa6-8164-4a61-8624-50975d678a12-kube-api-access-pjm4p\") pod \"82ea2aa6-8164-4a61-8624-50975d678a12\" (UID: \"82ea2aa6-8164-4a61-8624-50975d678a12\") " Sep 15 06:49:48.746889 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.746860 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ea2aa6-8164-4a61-8624-50975d678a12-feast-data" (OuterVolumeSpecName: "feast-data") pod "82ea2aa6-8164-4a61-8624-50975d678a12" (UID: "82ea2aa6-8164-4a61-8624-50975d678a12"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 06:49:48.748829 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.748802 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "82ea2aa6-8164-4a61-8624-50975d678a12" (UID: "82ea2aa6-8164-4a61-8624-50975d678a12"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:49:48.748921 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.748825 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "82ea2aa6-8164-4a61-8624-50975d678a12" (UID: "82ea2aa6-8164-4a61-8624-50975d678a12"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:49:48.748921 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.748797 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ea2aa6-8164-4a61-8624-50975d678a12-kube-api-access-pjm4p" (OuterVolumeSpecName: "kube-api-access-pjm4p") pod "82ea2aa6-8164-4a61-8624-50975d678a12" (UID: "82ea2aa6-8164-4a61-8624-50975d678a12"). InnerVolumeSpecName "kube-api-access-pjm4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:49:48.749058 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.749036 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "82ea2aa6-8164-4a61-8624-50975d678a12" (UID: "82ea2aa6-8164-4a61-8624-50975d678a12"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:49:48.749164 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.749122 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-online-tls" (OuterVolumeSpecName: "online-tls") pod "82ea2aa6-8164-4a61-8624-50975d678a12" (UID: "82ea2aa6-8164-4a61-8624-50975d678a12"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:49:48.847663 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.847609 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-offline-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:49:48.847663 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.847630 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjm4p\" (UniqueName: \"kubernetes.io/projected/82ea2aa6-8164-4a61-8624-50975d678a12-kube-api-access-pjm4p\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:49:48.847663 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.847640 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-ui-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:49:48.847663 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.847648 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-registry-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:49:48.847663 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.847657 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82ea2aa6-8164-4a61-8624-50975d678a12-feast-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:49:48.847663 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:48.847664 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82ea2aa6-8164-4a61-8624-50975d678a12-online-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:49:49.580880 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.580840 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" event={"ID":"82ea2aa6-8164-4a61-8624-50975d678a12","Type":"ContainerDied","Data":"f445d50a7ff2b660f27fae70f2552c9d7d897023e47307055c9d7e824c25e7f0"} Sep 15 06:49:49.580880 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.580884 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l" Sep 15 06:49:49.581418 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.580908 2581 scope.go:117] "RemoveContainer" containerID="645cc0f5685dddb87aeac91434d40509daee00fbd754c6d9091ebf6ce3f58895" Sep 15 06:49:49.597214 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.597189 2581 scope.go:117] "RemoveContainer" containerID="165373e8f751ba3569fab82c09bace189af333621c8c328a9c39ee5bcbf15c88" Sep 15 06:49:49.605545 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.605520 2581 scope.go:117] "RemoveContainer" containerID="ff418f091814049ea1a42dcacb6f15612c39b3e033aa250d77c54294e892ace2" Sep 15 06:49:49.607027 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.607006 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l"] Sep 15 06:49:49.609981 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.609958 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-2r97l"] Sep 15 06:49:49.615734 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.614865 2581 scope.go:117] "RemoveContainer" containerID="e0b3e16321a43e4f652b76c02f7523ad135c73e8ca24ef9930388acdba12f2f7" Sep 15 06:49:49.623769 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.623749 2581 scope.go:117] "RemoveContainer" containerID="87e27ac944046f3dd260509e01803ba1ee249e5ccc1abfc8f9bf472d78a2968f" Sep 15 06:49:49.630890 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:49.630873 2581 scope.go:117] "RemoveContainer" containerID="8722a6590873de0a65048c550e33ae0a8ac313857d298b4e74c0e89b9ffebb15" Sep 15 06:49:50.410070 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:49:50.410036 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" path="/var/lib/kubelet/pods/82ea2aa6-8164-4a61-8624-50975d678a12/volumes" Sep 15 06:50:42.845943 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:42.845901 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:50:42.949436 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:42.949401 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:50:43.275232 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.275188 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:50:43.362404 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.362325 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:50:43.362849 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.362827 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk"] Sep 15 06:50:43.363168 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363134 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="registry" Sep 15 06:50:43.363168 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363162 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="registry" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363171 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="feast-init" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363177 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="feast-init" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363189 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363194 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363200 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="offline" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363205 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="offline" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363218 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363223 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363232 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="feast-apply" Sep 15 06:50:43.363279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363237 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="feast-apply" Sep 15 06:50:43.363725 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363285 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="offline" Sep 15 06:50:43.363725 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363306 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="registry" Sep 15 06:50:43.363725 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363316 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="ui" Sep 15 06:50:43.363725 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.363326 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ea2aa6-8164-4a61-8624-50975d678a12" containerName="online" Sep 15 06:50:43.367817 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.367799 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.370097 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.370065 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 15 06:50:43.370097 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.370095 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 15 06:50:43.370299 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.370277 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 15 06:50:43.370398 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.370380 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 15 06:50:43.370727 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.370711 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 15 06:50:43.371282 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.371264 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-s65qx\"" Sep 15 06:50:43.371436 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.371307 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 15 06:50:43.371502 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.371314 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 15 06:50:43.377746 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.377270 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk"] Sep 15 06:50:43.420318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.420288 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e8ecbf33-e026-44fb-af1d-dbd2171c3479-registry-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.420475 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.420350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.420538 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.420517 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9pr\" (UniqueName: \"kubernetes.io/projected/e8ecbf33-e026-44fb-af1d-dbd2171c3479-kube-api-access-gg9pr\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.420598 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.420576 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-offline-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.420684 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.420668 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e8ecbf33-e026-44fb-af1d-dbd2171c3479-feast-data\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.420739 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.420702 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.517921 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.517883 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:50:43.521925 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.521903 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e8ecbf33-e026-44fb-af1d-dbd2171c3479-feast-data\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.522038 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.521938 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.522038 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.521967 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e8ecbf33-e026-44fb-af1d-dbd2171c3479-registry-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.522038 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.522000 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.522221 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:50:43.522057 2581 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 15 06:50:43.522221 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.522066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9pr\" (UniqueName: \"kubernetes.io/projected/e8ecbf33-e026-44fb-af1d-dbd2171c3479-kube-api-access-gg9pr\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.522221 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.522101 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-offline-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.522221 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:50:43.522132 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls podName:e8ecbf33-e026-44fb-af1d-dbd2171c3479 nodeName:}" failed. No retries permitted until 2026-09-15 06:50:44.022113489 +0000 UTC m=+852.096491039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls") pod "feast-simple-feast-remote-setup-68cb444cb5-qv9kk" (UID: "e8ecbf33-e026-44fb-af1d-dbd2171c3479") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 15 06:50:43.522221 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:50:43.522190 2581 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 15 06:50:43.522485 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:50:43.522263 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls podName:e8ecbf33-e026-44fb-af1d-dbd2171c3479 nodeName:}" failed. No retries permitted until 2026-09-15 06:50:44.022247318 +0000 UTC m=+852.096624849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls") pod "feast-simple-feast-remote-setup-68cb444cb5-qv9kk" (UID: "e8ecbf33-e026-44fb-af1d-dbd2171c3479") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 15 06:50:43.522485 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.522335 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e8ecbf33-e026-44fb-af1d-dbd2171c3479-feast-data\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.522773 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.522755 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e8ecbf33-e026-44fb-af1d-dbd2171c3479-registry-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.524792 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.524772 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-offline-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.530836 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.530774 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9pr\" (UniqueName: \"kubernetes.io/projected/e8ecbf33-e026-44fb-af1d-dbd2171c3479-kube-api-access-gg9pr\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:43.649859 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.649822 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:50:43.912595 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:43.912497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:50:44.027759 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:44.027723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:44.027953 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:44.027771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:44.030347 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:44.030322 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:44.030455 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:44.030427 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls\") pod \"feast-simple-feast-remote-setup-68cb444cb5-qv9kk\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:44.279532 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:44.279498 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:50:44.558576 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:44.558549 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk"] Sep 15 06:50:44.559300 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:50:44.559279 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8ecbf33_e026_44fb_af1d_dbd2171c3479.slice/crio-53f87c2f60cd11f8dd54130c8e386f1b7ceafd1c581b8b5373e783372c5173b6 WatchSource:0}: Error finding container 53f87c2f60cd11f8dd54130c8e386f1b7ceafd1c581b8b5373e783372c5173b6: Status 404 returned error can't find the container with id 53f87c2f60cd11f8dd54130c8e386f1b7ceafd1c581b8b5373e783372c5173b6 Sep 15 06:50:44.802833 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:44.802799 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerStarted","Data":"baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d"} Sep 15 06:50:44.803173 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:44.803133 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerStarted","Data":"53f87c2f60cd11f8dd54130c8e386f1b7ceafd1c581b8b5373e783372c5173b6"} Sep 15 06:50:49.823884 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:49.823854 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerID="baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d" exitCode=0 Sep 15 06:50:49.824254 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:49.823915 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerDied","Data":"baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d"} Sep 15 06:50:50.830619 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:50:50.830581 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerStarted","Data":"7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2"} Sep 15 06:51:11.924668 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:11.924634 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerID="7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2" exitCode=0 Sep 15 06:51:11.925128 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:11.924682 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerDied","Data":"7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2"} Sep 15 06:51:12.933426 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:12.933389 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerStarted","Data":"09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5"} Sep 15 06:51:12.933801 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:12.933454 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerStarted","Data":"bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f"} Sep 15 06:51:12.933801 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:12.933476 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerStarted","Data":"ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614"} Sep 15 06:51:12.956580 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:12.956208 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podStartSLOduration=29.956185656 podStartE2EDuration="29.956185656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:50:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:51:12.954773454 +0000 UTC m=+881.029151010" watchObservedRunningTime="2026-09-15 06:51:12.956185656 +0000 UTC m=+881.030563206" Sep 15 06:51:14.280103 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.280066 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:14.280570 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.280123 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:14.280570 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.280155 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:14.280570 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.280170 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:14.280570 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.280185 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:14.280570 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.280200 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:14.281893 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.281856 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 15 06:51:14.282023 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.281899 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Sep 15 06:51:14.282023 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:14.281899 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 15 06:51:17.280014 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:17.279967 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Sep 15 06:51:17.280506 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:17.280017 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 15 06:51:17.280506 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:17.280057 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 15 06:51:20.280841 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:20.280810 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:20.281273 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:20.281017 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:20.281656 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:20.281631 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:20.281893 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:20.281875 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:20.300740 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:20.300719 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:20.991478 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:20.991439 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:51:33.294075 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:51:33.294034 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:52:23.619043 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.619012 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk"] Sep 15 06:52:23.621685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.619486 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" containerID="cri-o://ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614" gracePeriod=30 Sep 15 06:52:23.621685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.619590 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="offline" containerID="cri-o://bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f" gracePeriod=30 Sep 15 06:52:23.621685 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.619619 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" containerID="cri-o://09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5" gracePeriod=30 Sep 15 06:52:23.624121 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.624089 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:52:23.727560 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.727527 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d"] Sep 15 06:52:23.727961 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.727777 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="registry" containerID="cri-o://9c753546f417e76ec2f4d2d8f45096e37ee52a228039d298f262ca27311e5d31" gracePeriod=30 Sep 15 06:52:23.727961 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.727811 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="offline" containerID="cri-o://73f6869337da9bf9c1b359738cce6e8638c5156fe2743216c87fe0bb3fa3b039" gracePeriod=30 Sep 15 06:52:23.727961 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.727849 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" containerID="cri-o://e4d9cc0571cfcb6406620258f2a6bddd5fd4ad131fa32834bb2ac09a7e17f74a" gracePeriod=30 Sep 15 06:52:23.728235 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:23.727946 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" containerID="cri-o://abdedf8110c22809e731ed5dc7498d58bdc9e435875a9d95bf3a7130d4c4588f" gracePeriod=30 Sep 15 06:52:25.236662 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:25.236631 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerID="09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5" exitCode=0 Sep 15 06:52:25.236662 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:25.236658 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerID="ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614" exitCode=0 Sep 15 06:52:25.237091 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:25.236706 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerDied","Data":"09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5"} Sep 15 06:52:25.237091 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:25.236750 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerDied","Data":"ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614"} Sep 15 06:52:25.239656 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:25.239636 2581 generic.go:356] "Generic (PLEG): container finished" podID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerID="e4d9cc0571cfcb6406620258f2a6bddd5fd4ad131fa32834bb2ac09a7e17f74a" exitCode=0 Sep 15 06:52:25.239656 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:25.239652 2581 generic.go:356] "Generic (PLEG): container finished" podID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerID="abdedf8110c22809e731ed5dc7498d58bdc9e435875a9d95bf3a7130d4c4588f" exitCode=0 Sep 15 06:52:25.239793 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:25.239682 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerDied","Data":"e4d9cc0571cfcb6406620258f2a6bddd5fd4ad131fa32834bb2ac09a7e17f74a"} Sep 15 06:52:25.239793 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:25.239705 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerDied","Data":"abdedf8110c22809e731ed5dc7498d58bdc9e435875a9d95bf3a7130d4c4588f"} Sep 15 06:52:27.187821 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:27.187776 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:52:28.892169 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:28.892105 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:52:30.281648 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:30.281596 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 15 06:52:30.306016 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:30.305981 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 15 06:52:30.538248 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:30.538126 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 15 06:52:30.974429 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:30.974341 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 15 06:52:40.281879 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:40.281831 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 15 06:52:40.305624 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:40.305583 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 15 06:52:40.537979 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:40.537884 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 15 06:52:40.974543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:40.974448 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 15 06:52:50.282578 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:50.282521 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 15 06:52:50.283033 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:50.282687 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:52:50.305502 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:50.305464 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 15 06:52:50.305656 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:50.305591 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:52:50.538515 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:50.538410 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 15 06:52:50.538678 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:50.538550 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:52:50.974515 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:50.974422 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 15 06:52:50.974688 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:50.974548 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:52:54.279121 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.279090 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:52:54.357728 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.357693 2581 generic.go:356] "Generic (PLEG): container finished" podID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerID="73f6869337da9bf9c1b359738cce6e8638c5156fe2743216c87fe0bb3fa3b039" exitCode=137 Sep 15 06:52:54.357728 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.357720 2581 generic.go:356] "Generic (PLEG): container finished" podID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerID="9c753546f417e76ec2f4d2d8f45096e37ee52a228039d298f262ca27311e5d31" exitCode=137 Sep 15 06:52:54.357955 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.357772 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerDied","Data":"73f6869337da9bf9c1b359738cce6e8638c5156fe2743216c87fe0bb3fa3b039"} Sep 15 06:52:54.357955 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.357829 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerDied","Data":"9c753546f417e76ec2f4d2d8f45096e37ee52a228039d298f262ca27311e5d31"} Sep 15 06:52:54.360752 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.360729 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerID="bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f" exitCode=137 Sep 15 06:52:54.360886 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.360778 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerDied","Data":"bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f"} Sep 15 06:52:54.360886 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.360812 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" Sep 15 06:52:54.360886 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.360826 2581 scope.go:117] "RemoveContainer" containerID="09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5" Sep 15 06:52:54.361046 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.360813 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk" event={"ID":"e8ecbf33-e026-44fb-af1d-dbd2171c3479","Type":"ContainerDied","Data":"53f87c2f60cd11f8dd54130c8e386f1b7ceafd1c581b8b5373e783372c5173b6"} Sep 15 06:52:54.370042 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.370026 2581 scope.go:117] "RemoveContainer" containerID="bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f" Sep 15 06:52:54.378871 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.378853 2581 scope.go:117] "RemoveContainer" containerID="ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614" Sep 15 06:52:54.380673 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.380651 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:52:54.386976 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.386957 2581 scope.go:117] "RemoveContainer" containerID="7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2" Sep 15 06:52:54.391209 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.391188 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e8ecbf33-e026-44fb-af1d-dbd2171c3479-registry-tls\") pod \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " Sep 15 06:52:54.391306 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.391221 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls\") pod \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " Sep 15 06:52:54.391306 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.391258 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls\") pod \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " Sep 15 06:52:54.391306 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.391283 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-offline-tls\") pod \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " Sep 15 06:52:54.391452 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.391339 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e8ecbf33-e026-44fb-af1d-dbd2171c3479-feast-data\") pod \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " Sep 15 06:52:54.391452 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.391374 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg9pr\" (UniqueName: \"kubernetes.io/projected/e8ecbf33-e026-44fb-af1d-dbd2171c3479-kube-api-access-gg9pr\") pod \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\" (UID: \"e8ecbf33-e026-44fb-af1d-dbd2171c3479\") " Sep 15 06:52:54.391692 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.391580 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8ecbf33-e026-44fb-af1d-dbd2171c3479-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e8ecbf33-e026-44fb-af1d-dbd2171c3479" (UID: "e8ecbf33-e026-44fb-af1d-dbd2171c3479"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:52:54.392043 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.391933 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8ecbf33-e026-44fb-af1d-dbd2171c3479-feast-data" (OuterVolumeSpecName: "feast-data") pod "e8ecbf33-e026-44fb-af1d-dbd2171c3479" (UID: "e8ecbf33-e026-44fb-af1d-dbd2171c3479"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 06:52:54.393588 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.393555 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e8ecbf33-e026-44fb-af1d-dbd2171c3479" (UID: "e8ecbf33-e026-44fb-af1d-dbd2171c3479"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:52:54.393714 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.393587 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e8ecbf33-e026-44fb-af1d-dbd2171c3479" (UID: "e8ecbf33-e026-44fb-af1d-dbd2171c3479"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:52:54.393978 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.393958 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls" (OuterVolumeSpecName: "online-tls") pod "e8ecbf33-e026-44fb-af1d-dbd2171c3479" (UID: "e8ecbf33-e026-44fb-af1d-dbd2171c3479"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:52:54.394034 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.393974 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8ecbf33-e026-44fb-af1d-dbd2171c3479-kube-api-access-gg9pr" (OuterVolumeSpecName: "kube-api-access-gg9pr") pod "e8ecbf33-e026-44fb-af1d-dbd2171c3479" (UID: "e8ecbf33-e026-44fb-af1d-dbd2171c3479"). InnerVolumeSpecName "kube-api-access-gg9pr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:52:54.394809 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.394793 2581 scope.go:117] "RemoveContainer" containerID="baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d" Sep 15 06:52:54.403395 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.403373 2581 scope.go:117] "RemoveContainer" containerID="09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5" Sep 15 06:52:54.403688 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:52:54.403663 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5\": container with ID starting with 09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5 not found: ID does not exist" containerID="09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5" Sep 15 06:52:54.403768 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.403699 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5"} err="failed to get container status \"09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5\": rpc error: code = NotFound desc = could not find container \"09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5\": container with ID starting with 09889dc9d365b2363fd8b400402e8e7ad49d1518ec333708f8bb36355e6cc6a5 not found: ID does not exist" Sep 15 06:52:54.403768 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.403726 2581 scope.go:117] "RemoveContainer" containerID="bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f" Sep 15 06:52:54.404038 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:52:54.404022 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f\": container with ID starting with bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f not found: ID does not exist" containerID="bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f" Sep 15 06:52:54.404093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.404042 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f"} err="failed to get container status \"bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f\": rpc error: code = NotFound desc = could not find container \"bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f\": container with ID starting with bb4a16556070e29d487cb2feb3f4b58a461a332d02ea809f2f11b92ee0f9814f not found: ID does not exist" Sep 15 06:52:54.404093 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.404055 2581 scope.go:117] "RemoveContainer" containerID="ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614" Sep 15 06:52:54.404564 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:52:54.404543 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614\": container with ID starting with ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614 not found: ID does not exist" containerID="ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614" Sep 15 06:52:54.404622 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.404571 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614"} err="failed to get container status \"ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614\": rpc error: code = NotFound desc = could not find container \"ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614\": container with ID starting with ab571e7543566d7469d7ca50878bfb5ff88b71da1fcafc08d6c274886380c614 not found: ID does not exist" Sep 15 06:52:54.404622 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.404590 2581 scope.go:117] "RemoveContainer" containerID="7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2" Sep 15 06:52:54.404883 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:52:54.404858 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2\": container with ID starting with 7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2 not found: ID does not exist" containerID="7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2" Sep 15 06:52:54.404981 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.404886 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2"} err="failed to get container status \"7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2\": rpc error: code = NotFound desc = could not find container \"7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2\": container with ID starting with 7f3d9b963bc824f0ab518142350c3ee303c401ac292e121a352fd66cfa1353f2 not found: ID does not exist" Sep 15 06:52:54.404981 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.404901 2581 scope.go:117] "RemoveContainer" containerID="baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d" Sep 15 06:52:54.405206 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:52:54.405177 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d\": container with ID starting with baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d not found: ID does not exist" containerID="baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d" Sep 15 06:52:54.405288 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.405212 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d"} err="failed to get container status \"baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d\": rpc error: code = NotFound desc = could not find container \"baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d\": container with ID starting with baa7958cc69182edf74e3d9107f746fa629bcb73d8e963f7a60722568d9aaa9d not found: ID does not exist" Sep 15 06:52:54.492088 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492054 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tpzm\" (UniqueName: \"kubernetes.io/projected/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-kube-api-access-7tpzm\") pod \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " Sep 15 06:52:54.492292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492105 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-online-tls\") pod \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " Sep 15 06:52:54.492292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492125 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-registry-tls\") pod \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " Sep 15 06:52:54.492292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492199 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-feast-data\") pod \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " Sep 15 06:52:54.492292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492216 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls\") pod \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " Sep 15 06:52:54.492292 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492232 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls\") pod \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\" (UID: \"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde\") " Sep 15 06:52:54.492543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492369 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e8ecbf33-e026-44fb-af1d-dbd2171c3479-registry-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.492543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492384 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-ui-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.492543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492398 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-online-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.492543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492411 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbf33-e026-44fb-af1d-dbd2171c3479-offline-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.492543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492422 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e8ecbf33-e026-44fb-af1d-dbd2171c3479-feast-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.492543 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492434 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg9pr\" (UniqueName: \"kubernetes.io/projected/e8ecbf33-e026-44fb-af1d-dbd2171c3479-kube-api-access-gg9pr\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.492814 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.492742 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-feast-data" (OuterVolumeSpecName: "feast-data") pod "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" (UID: "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 06:52:54.494393 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.494368 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" (UID: "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:52:54.494504 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.494479 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" (UID: "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:52:54.494550 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.494498 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-online-tls" (OuterVolumeSpecName: "online-tls") pod "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" (UID: "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:52:54.494848 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.494833 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" (UID: "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:52:54.494848 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.494841 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-kube-api-access-7tpzm" (OuterVolumeSpecName: "kube-api-access-7tpzm") pod "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" (UID: "1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde"). InnerVolumeSpecName "kube-api-access-7tpzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:52:54.592958 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.592930 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tpzm\" (UniqueName: \"kubernetes.io/projected/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-kube-api-access-7tpzm\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.592958 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.592952 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-online-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.592958 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.592961 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-registry-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.593208 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.592970 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-feast-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.593208 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.592977 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-offline-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.593208 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.592984 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde-ui-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:52:54.679279 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.679204 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk"] Sep 15 06:52:54.682980 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:54.682955 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68cb444cb5-qv9kk"] Sep 15 06:52:55.367771 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.367741 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" Sep 15 06:52:55.368337 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.367744 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d" event={"ID":"1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde","Type":"ContainerDied","Data":"7a1f4a35e2a3dfe12885c916b7da8cfa8b92ea2e0e8f9231a54691469788b9a5"} Sep 15 06:52:55.368337 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.367878 2581 scope.go:117] "RemoveContainer" containerID="e4d9cc0571cfcb6406620258f2a6bddd5fd4ad131fa32834bb2ac09a7e17f74a" Sep 15 06:52:55.376534 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.376520 2581 scope.go:117] "RemoveContainer" containerID="73f6869337da9bf9c1b359738cce6e8638c5156fe2743216c87fe0bb3fa3b039" Sep 15 06:52:55.384048 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.384028 2581 scope.go:117] "RemoveContainer" containerID="abdedf8110c22809e731ed5dc7498d58bdc9e435875a9d95bf3a7130d4c4588f" Sep 15 06:52:55.388777 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.388756 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d"] Sep 15 06:52:55.393197 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.393173 2581 scope.go:117] "RemoveContainer" containerID="9c753546f417e76ec2f4d2d8f45096e37ee52a228039d298f262ca27311e5d31" Sep 15 06:52:55.394424 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.393461 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b7c59ff95-wc58d"] Sep 15 06:52:55.402874 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.402849 2581 scope.go:117] "RemoveContainer" containerID="54b7cf0323617b643d533564b964eef433f61ce6fdf74d8194e861434fc77168" Sep 15 06:52:55.410454 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:55.410431 2581 scope.go:117] "RemoveContainer" containerID="add146156052169447243e53a3b2c96e5e1f4464853a0736a3828dbc981e3fd0" Sep 15 06:52:56.388904 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:56.388871 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:52:56.410124 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:56.410094 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" path="/var/lib/kubelet/pods/1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde/volumes" Sep 15 06:52:56.410745 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:52:56.410731 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" path="/var/lib/kubelet/pods/e8ecbf33-e026-44fb-af1d-dbd2171c3479/volumes" Sep 15 06:53:06.334091 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.334055 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:53:06.576621 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.576582 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-p6bm6"] Sep 15 06:53:06.576848 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.576812 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:53:06.577055 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577037 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" Sep 15 06:53:06.577115 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577061 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" Sep 15 06:53:06.577115 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577077 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" Sep 15 06:53:06.577115 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577085 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" Sep 15 06:53:06.577115 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577102 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="feast-init" Sep 15 06:53:06.577115 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577111 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="feast-init" Sep 15 06:53:06.577302 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577123 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" Sep 15 06:53:06.577302 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577131 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" Sep 15 06:53:06.577302 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577266 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="offline" Sep 15 06:53:06.577302 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577278 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="offline" Sep 15 06:53:06.577302 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577294 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="feast-apply" Sep 15 06:53:06.577302 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577302 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="feast-apply" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577316 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="feast-init" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577325 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="feast-init" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577343 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577352 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577362 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="registry" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577371 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="registry" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577383 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="feast-apply" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577392 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="feast-apply" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577400 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="offline" Sep 15 06:53:06.577474 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577408 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="offline" Sep 15 06:53:06.577759 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577494 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="ui" Sep 15 06:53:06.577759 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577505 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="registry" Sep 15 06:53:06.577759 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577516 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="online" Sep 15 06:53:06.577759 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577524 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="online" Sep 15 06:53:06.577759 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577532 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="offline" Sep 15 06:53:06.577759 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577540 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5350cb-8ea0-4987-9fdb-7c1edb0a6bde" containerName="ui" Sep 15 06:53:06.577759 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.577549 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ecbf33-e026-44fb-af1d-dbd2171c3479" containerName="offline" Sep 15 06:53:06.580632 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.580616 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-p6bm6" Sep 15 06:53:06.583095 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.583061 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-t7msp\"" Sep 15 06:53:06.583247 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.583135 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 15 06:53:06.583931 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.583903 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 15 06:53:06.585764 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.585696 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-p6bm6"] Sep 15 06:53:06.637009 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.636975 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:53:06.697221 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.697186 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgzn6\" (UniqueName: \"kubernetes.io/projected/430fa307-d1f7-4f69-ba7f-c0c71ab06e14-kube-api-access-cgzn6\") pod \"redis-69bbd798b6-p6bm6\" (UID: \"430fa307-d1f7-4f69-ba7f-c0c71ab06e14\") " pod="test-ns-feast/redis-69bbd798b6-p6bm6" Sep 15 06:53:06.798121 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.798086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgzn6\" (UniqueName: \"kubernetes.io/projected/430fa307-d1f7-4f69-ba7f-c0c71ab06e14-kube-api-access-cgzn6\") pod \"redis-69bbd798b6-p6bm6\" (UID: \"430fa307-d1f7-4f69-ba7f-c0c71ab06e14\") " pod="test-ns-feast/redis-69bbd798b6-p6bm6" Sep 15 06:53:06.806184 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.806134 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgzn6\" (UniqueName: \"kubernetes.io/projected/430fa307-d1f7-4f69-ba7f-c0c71ab06e14-kube-api-access-cgzn6\") pod \"redis-69bbd798b6-p6bm6\" (UID: \"430fa307-d1f7-4f69-ba7f-c0c71ab06e14\") " pod="test-ns-feast/redis-69bbd798b6-p6bm6" Sep 15 06:53:06.891191 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:06.891086 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-p6bm6" Sep 15 06:53:07.156917 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:07.156860 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-p6bm6"] Sep 15 06:53:07.159496 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:53:07.159468 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod430fa307_d1f7_4f69_ba7f_c0c71ab06e14.slice/crio-4093b7b58759ddc3fc076497aa31d85956dd0fa780d0fd8bb7dce0bc67a378e8 WatchSource:0}: Error finding container 4093b7b58759ddc3fc076497aa31d85956dd0fa780d0fd8bb7dce0bc67a378e8: Status 404 returned error can't find the container with id 4093b7b58759ddc3fc076497aa31d85956dd0fa780d0fd8bb7dce0bc67a378e8 Sep 15 06:53:07.161344 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:07.161326 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 06:53:07.421697 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:07.421610 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-p6bm6" event={"ID":"430fa307-d1f7-4f69-ba7f-c0c71ab06e14","Type":"ContainerStarted","Data":"4093b7b58759ddc3fc076497aa31d85956dd0fa780d0fd8bb7dce0bc67a378e8"} Sep 15 06:53:11.443161 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:11.443121 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-p6bm6" event={"ID":"430fa307-d1f7-4f69-ba7f-c0c71ab06e14","Type":"ContainerStarted","Data":"ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567"} Sep 15 06:53:11.458244 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:11.458198 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-p6bm6" podStartSLOduration=2.174946567 podStartE2EDuration="5.458181831s" totalImagesPullingTime="3.283235264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:53:11.45667053 +0000 UTC m=+999.531048095" watchObservedRunningTime="2026-09-15 06:53:11.458181831 +0000 UTC m=+999.532559380" Sep 15 06:53:17.285399 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.285360 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:53:17.377308 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.377273 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:53:17.379250 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.379224 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf"] Sep 15 06:53:17.383505 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.383487 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.385940 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.385916 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 15 06:53:17.386072 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.386037 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-xmqgn\"" Sep 15 06:53:17.386225 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.386186 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 15 06:53:17.386352 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.386268 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 15 06:53:17.386413 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.386360 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 15 06:53:17.386476 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.386433 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 15 06:53:17.393838 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.393816 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf"] Sep 15 06:53:17.467277 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.467247 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:53:17.485321 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.485285 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-online-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.485471 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.485332 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-registry-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.485471 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.485394 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-offline-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.485471 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.485437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ce614e-71e5-4ae1-acd6-739cfb293356-feast-data\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.485630 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.485477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bddvv\" (UniqueName: \"kubernetes.io/projected/a0ce614e-71e5-4ae1-acd6-739cfb293356-kube-api-access-bddvv\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.568882 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.568796 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:53:17.586552 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.586528 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bddvv\" (UniqueName: \"kubernetes.io/projected/a0ce614e-71e5-4ae1-acd6-739cfb293356-kube-api-access-bddvv\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.586620 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.586580 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-online-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.586712 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.586694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-registry-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.586746 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.586736 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-offline-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.586790 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.586779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ce614e-71e5-4ae1-acd6-739cfb293356-feast-data\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.587132 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.587106 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ce614e-71e5-4ae1-acd6-739cfb293356-feast-data\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.589218 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.589191 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-online-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.589314 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.589262 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-offline-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.589314 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.589297 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-registry-tls\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.594305 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.594281 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bddvv\" (UniqueName: \"kubernetes.io/projected/a0ce614e-71e5-4ae1-acd6-739cfb293356-kube-api-access-bddvv\") pod \"feast-credit-scoring-7947fdfc5b-8t5xf\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.669134 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.669104 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:53:17.696237 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.696201 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:17.971062 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:17.970985 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf"] Sep 15 06:53:17.973472 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:53:17.973444 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0ce614e_71e5_4ae1_acd6_739cfb293356.slice/crio-51aff5f7d59246213f27640325ce93b1cf07aef46441912221740c03b67a61af WatchSource:0}: Error finding container 51aff5f7d59246213f27640325ce93b1cf07aef46441912221740c03b67a61af: Status 404 returned error can't find the container with id 51aff5f7d59246213f27640325ce93b1cf07aef46441912221740c03b67a61af Sep 15 06:53:18.473501 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:18.473459 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerStarted","Data":"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e"} Sep 15 06:53:18.473895 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:18.473524 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerStarted","Data":"51aff5f7d59246213f27640325ce93b1cf07aef46441912221740c03b67a61af"} Sep 15 06:53:19.479233 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:19.479123 2581 generic.go:356] "Generic (PLEG): container finished" podID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerID="adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e" exitCode=0 Sep 15 06:53:19.479233 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:19.479186 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerDied","Data":"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e"} Sep 15 06:53:20.484869 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:20.484833 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerStarted","Data":"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15"} Sep 15 06:53:27.516572 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:27.516535 2581 generic.go:356] "Generic (PLEG): container finished" podID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerID="b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15" exitCode=0 Sep 15 06:53:27.517015 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:27.516612 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerDied","Data":"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15"} Sep 15 06:53:28.526649 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:28.526615 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerStarted","Data":"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09"} Sep 15 06:53:28.527244 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:28.526683 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerStarted","Data":"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08"} Sep 15 06:53:28.527244 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:28.526708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerStarted","Data":"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c"} Sep 15 06:53:28.549014 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:28.548957 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podStartSLOduration=11.548938265 podStartE2EDuration="11.548938265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:53:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:53:28.548595698 +0000 UTC m=+1016.622973249" watchObservedRunningTime="2026-09-15 06:53:28.548938265 +0000 UTC m=+1016.623315815" Sep 15 06:53:29.696411 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:29.696372 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:29.697005 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:29.696970 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:29.697130 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:29.697011 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:29.697921 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:29.697876 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 15 06:53:29.697921 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:29.697890 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 15 06:53:29.697921 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:29.697908 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 15 06:53:32.696944 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:32.696834 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 15 06:53:32.696944 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:32.696857 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 15 06:53:32.697489 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:32.696994 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 15 06:53:35.697369 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:35.697318 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 15 06:53:35.697956 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:35.697318 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 15 06:53:35.698197 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:35.697387 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 15 06:53:37.696609 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:37.696571 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:37.697097 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:37.696639 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:37.697097 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:37.696656 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:38.696884 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:38.696833 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 15 06:53:38.697735 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:38.697717 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:38.698524 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:38.698499 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:38.705107 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:38.705088 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:39.601050 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:39.601020 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:41.697417 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:41.697386 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:53:42.617623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:53:42.617599 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:54:43.324942 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:43.324907 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:54:48.311097 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.311060 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf"] Sep 15 06:54:48.311554 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.311316 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="registry" containerID="cri-o://8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c" gracePeriod=30 Sep 15 06:54:48.311554 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.311369 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="offline" containerID="cri-o://28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09" gracePeriod=30 Sep 15 06:54:48.311554 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.311408 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" containerID="cri-o://45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08" gracePeriod=30 Sep 15 06:54:48.338890 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.338861 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-p6bm6"] Sep 15 06:54:48.339054 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.339008 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-p6bm6" podUID="430fa307-d1f7-4f69-ba7f-c0c71ab06e14" containerName="redis" containerID="cri-o://ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567" gracePeriod=30 Sep 15 06:54:48.385867 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.385837 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:54:48.578088 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.578062 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-p6bm6" Sep 15 06:54:48.651418 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.651389 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgzn6\" (UniqueName: \"kubernetes.io/projected/430fa307-d1f7-4f69-ba7f-c0c71ab06e14-kube-api-access-cgzn6\") pod \"430fa307-d1f7-4f69-ba7f-c0c71ab06e14\" (UID: \"430fa307-d1f7-4f69-ba7f-c0c71ab06e14\") " Sep 15 06:54:48.653575 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.653539 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/430fa307-d1f7-4f69-ba7f-c0c71ab06e14-kube-api-access-cgzn6" (OuterVolumeSpecName: "kube-api-access-cgzn6") pod "430fa307-d1f7-4f69-ba7f-c0c71ab06e14" (UID: "430fa307-d1f7-4f69-ba7f-c0c71ab06e14"). InnerVolumeSpecName "kube-api-access-cgzn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:54:48.752051 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.752017 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgzn6\" (UniqueName: \"kubernetes.io/projected/430fa307-d1f7-4f69-ba7f-c0c71ab06e14-kube-api-access-cgzn6\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:54:48.901530 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.901443 2581 generic.go:356] "Generic (PLEG): container finished" podID="430fa307-d1f7-4f69-ba7f-c0c71ab06e14" containerID="ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567" exitCode=0 Sep 15 06:54:48.901530 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.901508 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-p6bm6" event={"ID":"430fa307-d1f7-4f69-ba7f-c0c71ab06e14","Type":"ContainerDied","Data":"ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567"} Sep 15 06:54:48.901530 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.901507 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-p6bm6" Sep 15 06:54:48.901762 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.901539 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-p6bm6" event={"ID":"430fa307-d1f7-4f69-ba7f-c0c71ab06e14","Type":"ContainerDied","Data":"4093b7b58759ddc3fc076497aa31d85956dd0fa780d0fd8bb7dce0bc67a378e8"} Sep 15 06:54:48.901762 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.901557 2581 scope.go:117] "RemoveContainer" containerID="ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567" Sep 15 06:54:48.909934 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.909915 2581 scope.go:117] "RemoveContainer" containerID="ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567" Sep 15 06:54:48.910227 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:54:48.910205 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567\": container with ID starting with ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567 not found: ID does not exist" containerID="ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567" Sep 15 06:54:48.910324 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.910249 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567"} err="failed to get container status \"ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567\": rpc error: code = NotFound desc = could not find container \"ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567\": container with ID starting with ef7f88244c05dcded71007ae436b18a6597dc857d357fe10bd28a78921e4c567 not found: ID does not exist" Sep 15 06:54:48.924635 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.924611 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-p6bm6"] Sep 15 06:54:48.926750 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:48.926727 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-p6bm6"] Sep 15 06:54:49.598369 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:49.598320 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 15 06:54:49.909558 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:49.909477 2581 generic.go:356] "Generic (PLEG): container finished" podID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerID="45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08" exitCode=0 Sep 15 06:54:49.909558 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:49.909513 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerDied","Data":"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08"} Sep 15 06:54:50.410636 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:50.410600 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="430fa307-d1f7-4f69-ba7f-c0c71ab06e14" path="/var/lib/kubelet/pods/430fa307-d1f7-4f69-ba7f-c0c71ab06e14/volumes" Sep 15 06:54:59.598125 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:54:59.598081 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 15 06:55:09.598277 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:09.598228 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 15 06:55:09.598696 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:09.598347 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:55:18.698518 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.698424 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 15 06:55:18.962189 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.962125 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:55:18.989136 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.989102 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-offline-tls\") pod \"a0ce614e-71e5-4ae1-acd6-739cfb293356\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " Sep 15 06:55:18.989299 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.989167 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-registry-tls\") pod \"a0ce614e-71e5-4ae1-acd6-739cfb293356\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " Sep 15 06:55:18.989299 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.989271 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bddvv\" (UniqueName: \"kubernetes.io/projected/a0ce614e-71e5-4ae1-acd6-739cfb293356-kube-api-access-bddvv\") pod \"a0ce614e-71e5-4ae1-acd6-739cfb293356\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " Sep 15 06:55:18.989429 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.989331 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ce614e-71e5-4ae1-acd6-739cfb293356-feast-data\") pod \"a0ce614e-71e5-4ae1-acd6-739cfb293356\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " Sep 15 06:55:18.989429 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.989412 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-online-tls\") pod \"a0ce614e-71e5-4ae1-acd6-739cfb293356\" (UID: \"a0ce614e-71e5-4ae1-acd6-739cfb293356\") " Sep 15 06:55:18.991761 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.991729 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0ce614e-71e5-4ae1-acd6-739cfb293356-kube-api-access-bddvv" (OuterVolumeSpecName: "kube-api-access-bddvv") pod "a0ce614e-71e5-4ae1-acd6-739cfb293356" (UID: "a0ce614e-71e5-4ae1-acd6-739cfb293356"). InnerVolumeSpecName "kube-api-access-bddvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:55:18.991885 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.991820 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a0ce614e-71e5-4ae1-acd6-739cfb293356" (UID: "a0ce614e-71e5-4ae1-acd6-739cfb293356"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:55:18.992080 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.992056 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-online-tls" (OuterVolumeSpecName: "online-tls") pod "a0ce614e-71e5-4ae1-acd6-739cfb293356" (UID: "a0ce614e-71e5-4ae1-acd6-739cfb293356"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:55:18.992176 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.992100 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a0ce614e-71e5-4ae1-acd6-739cfb293356" (UID: "a0ce614e-71e5-4ae1-acd6-739cfb293356"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:55:18.999014 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:18.998989 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ce614e-71e5-4ae1-acd6-739cfb293356-feast-data" (OuterVolumeSpecName: "feast-data") pod "a0ce614e-71e5-4ae1-acd6-739cfb293356" (UID: "a0ce614e-71e5-4ae1-acd6-739cfb293356"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 06:55:19.027379 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.027350 2581 generic.go:356] "Generic (PLEG): container finished" podID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerID="28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09" exitCode=137 Sep 15 06:55:19.027379 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.027374 2581 generic.go:356] "Generic (PLEG): container finished" podID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerID="8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c" exitCode=137 Sep 15 06:55:19.027524 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.027419 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" Sep 15 06:55:19.027524 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.027450 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerDied","Data":"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09"} Sep 15 06:55:19.027524 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.027495 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerDied","Data":"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c"} Sep 15 06:55:19.027524 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.027512 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf" event={"ID":"a0ce614e-71e5-4ae1-acd6-739cfb293356","Type":"ContainerDied","Data":"51aff5f7d59246213f27640325ce93b1cf07aef46441912221740c03b67a61af"} Sep 15 06:55:19.027654 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.027532 2581 scope.go:117] "RemoveContainer" containerID="28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09" Sep 15 06:55:19.036890 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.036872 2581 scope.go:117] "RemoveContainer" containerID="45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08" Sep 15 06:55:19.044718 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.044699 2581 scope.go:117] "RemoveContainer" containerID="8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c" Sep 15 06:55:19.048057 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.048034 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf"] Sep 15 06:55:19.051671 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.051650 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-7947fdfc5b-8t5xf"] Sep 15 06:55:19.052944 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.052931 2581 scope.go:117] "RemoveContainer" containerID="b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15" Sep 15 06:55:19.060169 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.060130 2581 scope.go:117] "RemoveContainer" containerID="adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e" Sep 15 06:55:19.067123 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.067107 2581 scope.go:117] "RemoveContainer" containerID="28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09" Sep 15 06:55:19.067418 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:55:19.067399 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09\": container with ID starting with 28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09 not found: ID does not exist" containerID="28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09" Sep 15 06:55:19.067488 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.067430 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09"} err="failed to get container status \"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09\": rpc error: code = NotFound desc = could not find container \"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09\": container with ID starting with 28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09 not found: ID does not exist" Sep 15 06:55:19.067488 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.067455 2581 scope.go:117] "RemoveContainer" containerID="45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08" Sep 15 06:55:19.067732 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:55:19.067715 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08\": container with ID starting with 45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08 not found: ID does not exist" containerID="45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08" Sep 15 06:55:19.067774 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.067738 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08"} err="failed to get container status \"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08\": rpc error: code = NotFound desc = could not find container \"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08\": container with ID starting with 45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08 not found: ID does not exist" Sep 15 06:55:19.067774 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.067753 2581 scope.go:117] "RemoveContainer" containerID="8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c" Sep 15 06:55:19.068000 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:55:19.067983 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c\": container with ID starting with 8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c not found: ID does not exist" containerID="8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c" Sep 15 06:55:19.068046 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.068005 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c"} err="failed to get container status \"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c\": rpc error: code = NotFound desc = could not find container \"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c\": container with ID starting with 8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c not found: ID does not exist" Sep 15 06:55:19.068046 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.068019 2581 scope.go:117] "RemoveContainer" containerID="b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15" Sep 15 06:55:19.068347 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:55:19.068319 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15\": container with ID starting with b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15 not found: ID does not exist" containerID="b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15" Sep 15 06:55:19.068411 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.068345 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15"} err="failed to get container status \"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15\": rpc error: code = NotFound desc = could not find container \"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15\": container with ID starting with b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15 not found: ID does not exist" Sep 15 06:55:19.068411 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.068360 2581 scope.go:117] "RemoveContainer" containerID="adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e" Sep 15 06:55:19.068605 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:55:19.068588 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e\": container with ID starting with adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e not found: ID does not exist" containerID="adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e" Sep 15 06:55:19.068645 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.068609 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e"} err="failed to get container status \"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e\": rpc error: code = NotFound desc = could not find container \"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e\": container with ID starting with adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e not found: ID does not exist" Sep 15 06:55:19.068645 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.068622 2581 scope.go:117] "RemoveContainer" containerID="28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09" Sep 15 06:55:19.068884 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.068866 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09"} err="failed to get container status \"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09\": rpc error: code = NotFound desc = could not find container \"28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09\": container with ID starting with 28209caa9c57370b0c9c406ce6b1615f8effaf3005eb47a6c8ce63d7f2d00f09 not found: ID does not exist" Sep 15 06:55:19.068926 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.068885 2581 scope.go:117] "RemoveContainer" containerID="45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08" Sep 15 06:55:19.069127 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.069108 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08"} err="failed to get container status \"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08\": rpc error: code = NotFound desc = could not find container \"45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08\": container with ID starting with 45d0b362b99afeb1082e640d69fef253d367b37b4459cca86ac18b6413c48a08 not found: ID does not exist" Sep 15 06:55:19.069127 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.069126 2581 scope.go:117] "RemoveContainer" containerID="8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c" Sep 15 06:55:19.069361 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.069346 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c"} err="failed to get container status \"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c\": rpc error: code = NotFound desc = could not find container \"8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c\": container with ID starting with 8a598685f3c36b45f450d5367aa43ccc9426f0dd3be2aa806f4f65ebf392439c not found: ID does not exist" Sep 15 06:55:19.069405 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.069361 2581 scope.go:117] "RemoveContainer" containerID="b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15" Sep 15 06:55:19.069590 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.069572 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15"} err="failed to get container status \"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15\": rpc error: code = NotFound desc = could not find container \"b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15\": container with ID starting with b2ddde982cc35e515468535004b851a43e9ddd4cbb8a911f54e99122e5a81b15 not found: ID does not exist" Sep 15 06:55:19.069644 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.069589 2581 scope.go:117] "RemoveContainer" containerID="adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e" Sep 15 06:55:19.069783 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.069767 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e"} err="failed to get container status \"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e\": rpc error: code = NotFound desc = could not find container \"adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e\": container with ID starting with adc2872468e8e45d8a063174b41d98d18752d5cfd6182c6b36086b73a680b67e not found: ID does not exist" Sep 15 06:55:19.091883 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.091864 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-registry-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:55:19.091883 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.091883 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bddvv\" (UniqueName: \"kubernetes.io/projected/a0ce614e-71e5-4ae1-acd6-739cfb293356-kube-api-access-bddvv\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:55:19.091973 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.091892 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ce614e-71e5-4ae1-acd6-739cfb293356-feast-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:55:19.091973 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.091900 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-online-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:55:19.091973 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:19.091908 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0ce614e-71e5-4ae1-acd6-739cfb293356-offline-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 06:55:20.410677 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:20.410641 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" path="/var/lib/kubelet/pods/a0ce614e-71e5-4ae1-acd6-739cfb293356/volumes" Sep 15 06:55:25.495937 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:25.495898 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:55:25.546765 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:25.546718 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:55:26.172226 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:26.172192 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:55:26.219416 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:26.219381 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:55:26.337117 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:55:26.337081 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:57:47.748177 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:47.748072 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:57:49.017690 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:49.017626 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:57:50.414659 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:50.414631 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:57:53.324488 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:53.324456 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:57:58.737542 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.737507 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:57:58.798586 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.798557 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:57:58.803180 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803137 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj"] Sep 15 06:57:58.803617 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803603 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="offline" Sep 15 06:57:58.803667 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803621 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="offline" Sep 15 06:57:58.803667 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803637 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="registry" Sep 15 06:57:58.803667 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803645 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="registry" Sep 15 06:57:58.803667 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803661 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="feast-init" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803669 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="feast-init" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803683 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="feast-apply" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803691 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="feast-apply" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803702 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803710 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803722 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="430fa307-d1f7-4f69-ba7f-c0c71ab06e14" containerName="redis" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803730 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="430fa307-d1f7-4f69-ba7f-c0c71ab06e14" containerName="redis" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803801 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="430fa307-d1f7-4f69-ba7f-c0c71ab06e14" containerName="redis" Sep 15 06:57:58.803810 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803811 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="registry" Sep 15 06:57:58.804116 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803821 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="offline" Sep 15 06:57:58.804116 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.803830 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ce614e-71e5-4ae1-acd6-739cfb293356" containerName="online" Sep 15 06:57:58.807261 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.807243 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.809724 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.809550 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 15 06:57:58.809724 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.809569 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 15 06:57:58.809724 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.809580 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 15 06:57:58.809724 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.809679 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-dbm5r\"" Sep 15 06:57:58.810029 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.809874 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 15 06:57:58.810029 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.809892 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 15 06:57:58.810029 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.809910 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 15 06:57:58.819699 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.819678 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj"] Sep 15 06:57:58.861238 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.861211 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jprq\" (UniqueName: \"kubernetes.io/projected/b0fd9d10-888c-46c4-a897-80df9e3f4197-kube-api-access-9jprq\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.861387 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.861244 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.861387 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.861265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.861483 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.861416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0fd9d10-888c-46c4-a897-80df9e3f4197-feast-data\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.962072 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.962044 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0fd9d10-888c-46c4-a897-80df9e3f4197-feast-data\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.962237 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.962083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jprq\" (UniqueName: \"kubernetes.io/projected/b0fd9d10-888c-46c4-a897-80df9e3f4197-kube-api-access-9jprq\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.962237 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.962107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.962237 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.962127 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.962418 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:57:58.962250 2581 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 15 06:57:58.962418 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:57:58.962250 2581 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 15 06:57:58.962418 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:57:58.962328 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls podName:b0fd9d10-888c-46c4-a897-80df9e3f4197 nodeName:}" failed. No retries permitted until 2026-09-15 06:57:59.462308868 +0000 UTC m=+1287.536686396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls") pod "feast-credit-scoring-7d87c67789-72fzj" (UID: "b0fd9d10-888c-46c4-a897-80df9e3f4197") : secret "feast-credit-scoring-online-tls" not found Sep 15 06:57:58.962418 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:57:58.962366 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls podName:b0fd9d10-888c-46c4-a897-80df9e3f4197 nodeName:}" failed. No retries permitted until 2026-09-15 06:57:59.462338751 +0000 UTC m=+1287.536716295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls") pod "feast-credit-scoring-7d87c67789-72fzj" (UID: "b0fd9d10-888c-46c4-a897-80df9e3f4197") : secret "feast-credit-scoring-registry-tls" not found Sep 15 06:57:58.962611 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.962510 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0fd9d10-888c-46c4-a897-80df9e3f4197-feast-data\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:58.972815 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:58.972792 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jprq\" (UniqueName: \"kubernetes.io/projected/b0fd9d10-888c-46c4-a897-80df9e3f4197-kube-api-access-9jprq\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:59.121130 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:59.121104 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:57:59.466648 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:59.466570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:59.466648 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:59.466605 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:59.466823 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:57:59.466711 2581 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 15 06:57:59.466823 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:57:59.466768 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls podName:b0fd9d10-888c-46c4-a897-80df9e3f4197 nodeName:}" failed. No retries permitted until 2026-09-15 06:58:00.46675383 +0000 UTC m=+1288.541131359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls") pod "feast-credit-scoring-7d87c67789-72fzj" (UID: "b0fd9d10-888c-46c4-a897-80df9e3f4197") : secret "feast-credit-scoring-online-tls" not found Sep 15 06:57:59.469074 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:59.469047 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:57:59.836443 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:57:59.836414 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:58:00.475323 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:00.475285 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:00.477780 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:00.477761 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls\") pod \"feast-credit-scoring-7d87c67789-72fzj\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:00.616883 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:00.616852 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:00.889452 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:00.889425 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj"] Sep 15 06:58:00.890779 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:58:00.890738 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0fd9d10_888c_46c4_a897_80df9e3f4197.slice/crio-96622251052deca724896ca5c466ff03538a43258a7bad1ce058a2bb31f8d0f6 WatchSource:0}: Error finding container 96622251052deca724896ca5c466ff03538a43258a7bad1ce058a2bb31f8d0f6: Status 404 returned error can't find the container with id 96622251052deca724896ca5c466ff03538a43258a7bad1ce058a2bb31f8d0f6 Sep 15 06:58:01.642013 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:01.641977 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerStarted","Data":"3a106a4e03365ac5c033a37166d79cf49cb8b95a0bc7597758ffc3bf0f7f62da"} Sep 15 06:58:01.642206 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:01.642041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerStarted","Data":"96622251052deca724896ca5c466ff03538a43258a7bad1ce058a2bb31f8d0f6"} Sep 15 06:58:02.646736 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:02.646698 2581 generic.go:356] "Generic (PLEG): container finished" podID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerID="3a106a4e03365ac5c033a37166d79cf49cb8b95a0bc7597758ffc3bf0f7f62da" exitCode=0 Sep 15 06:58:02.647133 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:02.646763 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerDied","Data":"3a106a4e03365ac5c033a37166d79cf49cb8b95a0bc7597758ffc3bf0f7f62da"} Sep 15 06:58:03.653814 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:03.653775 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerStarted","Data":"83fcd67152e4df782e654921ceb2c99260f8276661a418f4c758fc33b6fb723e"} Sep 15 06:58:10.685925 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:10.685894 2581 generic.go:356] "Generic (PLEG): container finished" podID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerID="83fcd67152e4df782e654921ceb2c99260f8276661a418f4c758fc33b6fb723e" exitCode=0 Sep 15 06:58:10.686318 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:10.685938 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerDied","Data":"83fcd67152e4df782e654921ceb2c99260f8276661a418f4c758fc33b6fb723e"} Sep 15 06:58:11.693948 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:11.693911 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerStarted","Data":"5ab36e09f5be1c6be74c803315e06cbdab55f59dfac1bcd92906f3e7883a2601"} Sep 15 06:58:11.694475 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:11.693963 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerStarted","Data":"cf3c8e5c90ba34721fc9face450ebeb1f7ceacbd3d7b851ef145a687220d293a"} Sep 15 06:58:11.714600 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:11.714549 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podStartSLOduration=13.714532404 podStartE2EDuration="13.714532404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:58:11.712522068 +0000 UTC m=+1299.786899618" watchObservedRunningTime="2026-09-15 06:58:11.714532404 +0000 UTC m=+1299.788909953" Sep 15 06:58:12.617398 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:12.617364 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:12.617584 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:12.617432 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:12.618604 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:12.618572 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 15 06:58:12.618604 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:12.618585 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused" Sep 15 06:58:15.617916 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:15.617867 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused" Sep 15 06:58:15.618404 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:15.617879 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 15 06:58:18.617876 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:18.617823 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused" Sep 15 06:58:18.626025 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:18.625995 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:18.626481 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:18.626462 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:18.630625 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:18.630601 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:20.617402 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:20.617368 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:21.617515 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:21.617471 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused" Sep 15 06:58:24.618418 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:24.618386 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:24.619220 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:24.619199 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 06:58:29.672807 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.672765 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:58:29.749365 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.749323 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:58:29.749587 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.749395 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr"] Sep 15 06:58:29.754514 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.754493 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.756773 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.756739 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 15 06:58:29.756773 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.756766 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 15 06:58:29.756984 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.756739 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 15 06:58:29.756984 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.756739 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-8nn4t\"" Sep 15 06:58:29.761658 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.761634 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr"] Sep 15 06:58:29.809467 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.809438 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:58:29.821854 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.821821 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dda725-d729-43b8-b25a-f87840c059e6-feast-data\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.821854 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.821855 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.822035 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.821876 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97cpq\" (UniqueName: \"kubernetes.io/projected/e7dda725-d729-43b8-b25a-f87840c059e6-kube-api-access-97cpq\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.822072 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.822022 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-online-tls\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.922481 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.922444 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97cpq\" (UniqueName: \"kubernetes.io/projected/e7dda725-d729-43b8-b25a-f87840c059e6-kube-api-access-97cpq\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.922623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.922507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-online-tls\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.922623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.922539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dda725-d729-43b8-b25a-f87840c059e6-feast-data\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.922623 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.922556 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.922720 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:58:29.922651 2581 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found Sep 15 06:58:29.922765 ip-10-0-128-50 kubenswrapper[2581]: E0915 06:58:29.922729 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls podName:e7dda725-d729-43b8-b25a-f87840c059e6 nodeName:}" failed. No retries permitted until 2026-09-15 06:58:30.42270888 +0000 UTC m=+1318.497086410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls") pod "feast-driver-ranking-6689cf6874-wh9vr" (UID: "e7dda725-d729-43b8-b25a-f87840c059e6") : secret "feast-driver-ranking-registry-tls" not found Sep 15 06:58:29.923031 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.922986 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dda725-d729-43b8-b25a-f87840c059e6-feast-data\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.925418 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.925392 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-online-tls\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:29.940715 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:29.940681 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97cpq\" (UniqueName: \"kubernetes.io/projected/e7dda725-d729-43b8-b25a-f87840c059e6-kube-api-access-97cpq\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:30.143614 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:30.143581 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 06:58:30.426502 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:30.426460 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:30.429124 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:30.429103 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls\") pod \"feast-driver-ranking-6689cf6874-wh9vr\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:30.666411 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:30.666367 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:30.942416 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:30.942388 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr"] Sep 15 06:58:30.943762 ip-10-0-128-50 kubenswrapper[2581]: W0915 06:58:30.943729 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7dda725_d729_43b8_b25a_f87840c059e6.slice/crio-5331eb4b1ea27a7d0c42a2c375e6e9151c08be22f31cb7ebd1db9eed66dcdaba WatchSource:0}: Error finding container 5331eb4b1ea27a7d0c42a2c375e6e9151c08be22f31cb7ebd1db9eed66dcdaba: Status 404 returned error can't find the container with id 5331eb4b1ea27a7d0c42a2c375e6e9151c08be22f31cb7ebd1db9eed66dcdaba Sep 15 06:58:31.781167 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:31.781101 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerStarted","Data":"f805947016075f78dabf6bf3c6a9d89a8eadeaf488f9e73f53319dc8f5e01088"} Sep 15 06:58:31.781347 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:31.781200 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerStarted","Data":"5331eb4b1ea27a7d0c42a2c375e6e9151c08be22f31cb7ebd1db9eed66dcdaba"} Sep 15 06:58:36.804488 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:36.804450 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7dda725-d729-43b8-b25a-f87840c059e6" containerID="f805947016075f78dabf6bf3c6a9d89a8eadeaf488f9e73f53319dc8f5e01088" exitCode=0 Sep 15 06:58:36.804917 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:36.804532 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerDied","Data":"f805947016075f78dabf6bf3c6a9d89a8eadeaf488f9e73f53319dc8f5e01088"} Sep 15 06:58:37.811546 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:37.811507 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerStarted","Data":"7a13ad5d23782b74c18a020c937b896f7964b480061673e3e57ab091f5b87e01"} Sep 15 06:58:44.843263 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:44.843226 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7dda725-d729-43b8-b25a-f87840c059e6" containerID="7a13ad5d23782b74c18a020c937b896f7964b480061673e3e57ab091f5b87e01" exitCode=0 Sep 15 06:58:44.843636 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:44.843289 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerDied","Data":"7a13ad5d23782b74c18a020c937b896f7964b480061673e3e57ab091f5b87e01"} Sep 15 06:58:44.847589 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:44.847565 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 15 06:58:45.851384 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:45.851349 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerStarted","Data":"372232cf931d011264d72922deb07dcf44ea03f7d1ff08ce0e63c9e6fef0bac4"} Sep 15 06:58:45.851824 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:45.851407 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerStarted","Data":"1db65ffadd23a4b381f6db1e446462432fd347810fa967c721d8693806c092e3"} Sep 15 06:58:45.873824 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:45.873761 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podStartSLOduration=16.873740959 podStartE2EDuration="16.873740959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:58:45.871573356 +0000 UTC m=+1333.945950910" watchObservedRunningTime="2026-09-15 06:58:45.873740959 +0000 UTC m=+1333.948118510" Sep 15 06:58:48.666753 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:48.666710 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:48.667285 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:48.666785 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:48.668381 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:48.668343 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused" Sep 15 06:58:48.668505 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:48.668350 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 15 06:58:50.666781 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:50.666741 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:50.667180 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:50.666814 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:51.667342 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:51.667291 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused" Sep 15 06:58:51.673329 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:51.673297 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:51.678041 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:51.677996 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:54.667275 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:54.667181 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused" Sep 15 06:58:57.667429 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:57.667397 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 06:58:57.668225 ip-10-0-128-50 kubenswrapper[2581]: I0915 06:58:57.668203 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 07:00:16.164882 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:16.164842 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:18.178048 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.178011 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:18.506031 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.505992 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:18.780673 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.780589 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:18.806335 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.806294 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj"] Sep 15 07:00:18.812092 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.812048 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="registry" containerID="cri-o://cf3c8e5c90ba34721fc9face450ebeb1f7ceacbd3d7b851ef145a687220d293a" gracePeriod=30 Sep 15 07:00:18.812346 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.812310 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" containerID="cri-o://5ab36e09f5be1c6be74c803315e06cbdab55f59dfac1bcd92906f3e7883a2601" gracePeriod=30 Sep 15 07:00:18.818001 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.817971 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr"] Sep 15 07:00:18.818268 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.818239 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="registry" containerID="cri-o://1db65ffadd23a4b381f6db1e446462432fd347810fa967c721d8693806c092e3" gracePeriod=30 Sep 15 07:00:18.818386 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.818287 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="online" containerID="cri-o://372232cf931d011264d72922deb07dcf44ea03f7d1ff08ce0e63c9e6fef0bac4" gracePeriod=30 Sep 15 07:00:18.879586 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:18.879550 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:20.277599 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:20.277566 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7dda725-d729-43b8-b25a-f87840c059e6" containerID="372232cf931d011264d72922deb07dcf44ea03f7d1ff08ce0e63c9e6fef0bac4" exitCode=0 Sep 15 07:00:20.277978 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:20.277634 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerDied","Data":"372232cf931d011264d72922deb07dcf44ea03f7d1ff08ce0e63c9e6fef0bac4"} Sep 15 07:00:20.279972 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:20.279951 2581 generic.go:356] "Generic (PLEG): container finished" podID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerID="5ab36e09f5be1c6be74c803315e06cbdab55f59dfac1bcd92906f3e7883a2601" exitCode=0 Sep 15 07:00:20.280068 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:20.279993 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerDied","Data":"5ab36e09f5be1c6be74c803315e06cbdab55f59dfac1bcd92906f3e7883a2601"} Sep 15 07:00:21.127875 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:21.127840 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:21.674290 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:21.674249 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 15 07:00:23.612464 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:23.612427 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:23.664157 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:23.664124 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:28.627357 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:28.627310 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 15 07:00:29.689186 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:29.689123 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:29.786986 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:29.786950 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:30.194643 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:30.194612 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:31.674496 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:31.674446 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 15 07:00:38.627236 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:38.627192 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 15 07:00:41.674412 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:41.674370 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 15 07:00:41.674824 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:41.674484 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 07:00:46.101857 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:46.101818 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:46.390476 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:46.390374 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:46.493173 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:46.493112 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:46.957726 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:46.957677 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:47.060695 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:47.060656 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:47.157919 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:47.157882 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:47.258819 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:47.258778 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:47.359350 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:47.359314 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:47.475418 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:47.475373 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:48.626988 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:48.626944 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 15 07:00:48.627477 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:48.627083 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 07:00:49.406581 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.406548 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7dda725-d729-43b8-b25a-f87840c059e6" containerID="1db65ffadd23a4b381f6db1e446462432fd347810fa967c721d8693806c092e3" exitCode=137 Sep 15 07:00:49.406763 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.406615 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerDied","Data":"1db65ffadd23a4b381f6db1e446462432fd347810fa967c721d8693806c092e3"} Sep 15 07:00:49.409097 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.409074 2581 generic.go:356] "Generic (PLEG): container finished" podID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerID="cf3c8e5c90ba34721fc9face450ebeb1f7ceacbd3d7b851ef145a687220d293a" exitCode=137 Sep 15 07:00:49.409249 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.409170 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerDied","Data":"cf3c8e5c90ba34721fc9face450ebeb1f7ceacbd3d7b851ef145a687220d293a"} Sep 15 07:00:49.486265 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.486237 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 07:00:49.488894 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.488870 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 07:00:49.631356 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.631266 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jprq\" (UniqueName: \"kubernetes.io/projected/b0fd9d10-888c-46c4-a897-80df9e3f4197-kube-api-access-9jprq\") pod \"b0fd9d10-888c-46c4-a897-80df9e3f4197\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " Sep 15 07:00:49.631356 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.631311 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls\") pod \"e7dda725-d729-43b8-b25a-f87840c059e6\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " Sep 15 07:00:49.631356 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.631338 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-online-tls\") pod \"e7dda725-d729-43b8-b25a-f87840c059e6\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " Sep 15 07:00:49.631991 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.631488 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls\") pod \"b0fd9d10-888c-46c4-a897-80df9e3f4197\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " Sep 15 07:00:49.631991 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.631525 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97cpq\" (UniqueName: \"kubernetes.io/projected/e7dda725-d729-43b8-b25a-f87840c059e6-kube-api-access-97cpq\") pod \"e7dda725-d729-43b8-b25a-f87840c059e6\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " Sep 15 07:00:49.631991 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.631592 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0fd9d10-888c-46c4-a897-80df9e3f4197-feast-data\") pod \"b0fd9d10-888c-46c4-a897-80df9e3f4197\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " Sep 15 07:00:49.631991 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.631649 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls\") pod \"b0fd9d10-888c-46c4-a897-80df9e3f4197\" (UID: \"b0fd9d10-888c-46c4-a897-80df9e3f4197\") " Sep 15 07:00:49.631991 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.631699 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dda725-d729-43b8-b25a-f87840c059e6-feast-data\") pod \"e7dda725-d729-43b8-b25a-f87840c059e6\" (UID: \"e7dda725-d729-43b8-b25a-f87840c059e6\") " Sep 15 07:00:49.632578 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.632524 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7dda725-d729-43b8-b25a-f87840c059e6-feast-data" (OuterVolumeSpecName: "feast-data") pod "e7dda725-d729-43b8-b25a-f87840c059e6" (UID: "e7dda725-d729-43b8-b25a-f87840c059e6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:00:49.634329 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.634288 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0fd9d10-888c-46c4-a897-80df9e3f4197-kube-api-access-9jprq" (OuterVolumeSpecName: "kube-api-access-9jprq") pod "b0fd9d10-888c-46c4-a897-80df9e3f4197" (UID: "b0fd9d10-888c-46c4-a897-80df9e3f4197"). InnerVolumeSpecName "kube-api-access-9jprq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:00:49.634433 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.634350 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e7dda725-d729-43b8-b25a-f87840c059e6" (UID: "e7dda725-d729-43b8-b25a-f87840c059e6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:00:49.634433 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.634363 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b0fd9d10-888c-46c4-a897-80df9e3f4197" (UID: "b0fd9d10-888c-46c4-a897-80df9e3f4197"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:00:49.634648 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.634626 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-online-tls" (OuterVolumeSpecName: "online-tls") pod "e7dda725-d729-43b8-b25a-f87840c059e6" (UID: "e7dda725-d729-43b8-b25a-f87840c059e6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:00:49.634702 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.634623 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls" (OuterVolumeSpecName: "online-tls") pod "b0fd9d10-888c-46c4-a897-80df9e3f4197" (UID: "b0fd9d10-888c-46c4-a897-80df9e3f4197"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:00:49.634702 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.634630 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7dda725-d729-43b8-b25a-f87840c059e6-kube-api-access-97cpq" (OuterVolumeSpecName: "kube-api-access-97cpq") pod "e7dda725-d729-43b8-b25a-f87840c059e6" (UID: "e7dda725-d729-43b8-b25a-f87840c059e6"). InnerVolumeSpecName "kube-api-access-97cpq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:00:49.642226 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.642171 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fd9d10-888c-46c4-a897-80df9e3f4197-feast-data" (OuterVolumeSpecName: "feast-data") pod "b0fd9d10-888c-46c4-a897-80df9e3f4197" (UID: "b0fd9d10-888c-46c4-a897-80df9e3f4197"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:00:49.732700 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.732663 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0fd9d10-888c-46c4-a897-80df9e3f4197-feast-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 07:00:49.732700 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.732696 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-online-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 07:00:49.732700 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.732705 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dda725-d729-43b8-b25a-f87840c059e6-feast-data\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 07:00:49.732955 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.732713 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jprq\" (UniqueName: \"kubernetes.io/projected/b0fd9d10-888c-46c4-a897-80df9e3f4197-kube-api-access-9jprq\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 07:00:49.732955 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.732722 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-registry-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 07:00:49.732955 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.732730 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dda725-d729-43b8-b25a-f87840c059e6-online-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 07:00:49.732955 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.732737 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b0fd9d10-888c-46c4-a897-80df9e3f4197-registry-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 07:00:49.732955 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:49.732745 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97cpq\" (UniqueName: \"kubernetes.io/projected/e7dda725-d729-43b8-b25a-f87840c059e6-kube-api-access-97cpq\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Sep 15 07:00:50.416326 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.416293 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" Sep 15 07:00:50.416326 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.416307 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr" event={"ID":"e7dda725-d729-43b8-b25a-f87840c059e6","Type":"ContainerDied","Data":"5331eb4b1ea27a7d0c42a2c375e6e9151c08be22f31cb7ebd1db9eed66dcdaba"} Sep 15 07:00:50.416596 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.416389 2581 scope.go:117] "RemoveContainer" containerID="372232cf931d011264d72922deb07dcf44ea03f7d1ff08ce0e63c9e6fef0bac4" Sep 15 07:00:50.419698 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.419654 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" event={"ID":"b0fd9d10-888c-46c4-a897-80df9e3f4197","Type":"ContainerDied","Data":"96622251052deca724896ca5c466ff03538a43258a7bad1ce058a2bb31f8d0f6"} Sep 15 07:00:50.419806 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.419721 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj" Sep 15 07:00:50.431050 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.430979 2581 scope.go:117] "RemoveContainer" containerID="1db65ffadd23a4b381f6db1e446462432fd347810fa967c721d8693806c092e3" Sep 15 07:00:50.440169 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.440124 2581 scope.go:117] "RemoveContainer" containerID="7a13ad5d23782b74c18a020c937b896f7964b480061673e3e57ab091f5b87e01" Sep 15 07:00:50.441179 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.441135 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj"] Sep 15 07:00:50.444659 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.444639 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d87c67789-72fzj"] Sep 15 07:00:50.448375 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.448359 2581 scope.go:117] "RemoveContainer" containerID="f805947016075f78dabf6bf3c6a9d89a8eadeaf488f9e73f53319dc8f5e01088" Sep 15 07:00:50.455951 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.455927 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr"] Sep 15 07:00:50.456325 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.456304 2581 scope.go:117] "RemoveContainer" containerID="5ab36e09f5be1c6be74c803315e06cbdab55f59dfac1bcd92906f3e7883a2601" Sep 15 07:00:50.460122 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.460102 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6689cf6874-wh9vr"] Sep 15 07:00:50.464454 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.464432 2581 scope.go:117] "RemoveContainer" containerID="cf3c8e5c90ba34721fc9face450ebeb1f7ceacbd3d7b851ef145a687220d293a" Sep 15 07:00:50.472589 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.472567 2581 scope.go:117] "RemoveContainer" containerID="83fcd67152e4df782e654921ceb2c99260f8276661a418f4c758fc33b6fb723e" Sep 15 07:00:50.480360 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:50.480335 2581 scope.go:117] "RemoveContainer" containerID="3a106a4e03365ac5c033a37166d79cf49cb8b95a0bc7597758ffc3bf0f7f62da" Sep 15 07:00:52.411047 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:52.411010 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" path="/var/lib/kubelet/pods/b0fd9d10-888c-46c4-a897-80df9e3f4197/volumes" Sep 15 07:00:52.411568 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:52.411554 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" path="/var/lib/kubelet/pods/e7dda725-d729-43b8-b25a-f87840c059e6/volumes" Sep 15 07:00:58.607903 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:58.607871 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:58.659837 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:58.659806 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:00:58.745206 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:00:58.745167 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:01:36.512720 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:01:36.512685 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:01:36.597301 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:01:36.597249 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:02:25.380751 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:02:25.380699 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:02:25.479062 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:02:25.479025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:02:25.676884 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:02:25.676789 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:02:25.775004 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:02:25.774959 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:02:25.938648 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:02:25.938549 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:02:26.138516 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:02:26.138479 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:02:26.235919 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:02:26.235838 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:02:26.811974 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:02:26.811925 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:03:52.283459 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:03:52.283427 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:03:56.044266 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:03:56.044216 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:04:51.852331 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:04:51.852297 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:05:39.224575 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:05:39.224533 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:05:39.326897 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:05:39.326859 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:05:39.433889 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:05:39.433835 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:05:39.485418 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:05:39.485328 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:05:44.521865 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:05:44.521827 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:08.532884 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:08.532844 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:21.104409 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:21.104372 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:21.220797 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:21.220758 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:26.708721 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:26.708644 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:26.808920 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:26.808880 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:47.692695 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:47.692659 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:47.795984 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:47.795952 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:48.894728 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:48.894684 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:48.995757 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:48.995721 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:49.106505 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:49.106459 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:49.229705 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:49.229597 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:06:49.504155 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:06:49.504122 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:07:01.944836 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:07:01.944801 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:08:16.424605 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:08:16.424573 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:08:16.490312 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:08:16.490277 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:08:16.735593 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:08:16.735506 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:08:16.835698 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:08:16.835659 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:08:16.942488 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:08:16.942454 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:08:17.041749 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:08:17.041718 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:09:40.968922 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:09:40.968878 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:09:44.110954 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:09:44.110923 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:11:23.610769 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:11:23.610730 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:11:23.663560 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:11:23.663513 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:11:23.730096 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:11:23.730062 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:11:23.830020 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:11:23.829987 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:11:23.930927 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:11:23.930838 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:11:28.896179 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:11:28.896109 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:11:31.726805 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:11:31.726771 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:11:31.795822 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:11:31.795792 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:06.416058 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:06.416023 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:06.543432 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:06.543399 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:12.206267 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:12.206227 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:12.259110 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:12.259078 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:13.294516 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:13.294480 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:13.423539 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:13.423503 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:16.515293 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:16.515236 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:30.872899 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.872827 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:30.992979 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.992940 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kgbcs/must-gather-wv4lh"] Sep 15 07:12:30.993332 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993315 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="feast-apply" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993334 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="feast-apply" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993347 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993355 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993363 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="feast-apply" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993372 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="feast-apply" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993386 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="registry" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993395 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="registry" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993404 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="feast-init" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993412 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="feast-init" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993429 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="registry" Sep 15 07:12:30.993434 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993436 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="registry" Sep 15 07:12:30.994022 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993447 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="online" Sep 15 07:12:30.994022 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993455 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="online" Sep 15 07:12:30.994022 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993477 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="feast-init" Sep 15 07:12:30.994022 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993485 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="feast-init" Sep 15 07:12:30.994022 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993560 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="registry" Sep 15 07:12:30.994022 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993576 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="online" Sep 15 07:12:30.994022 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993584 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dda725-d729-43b8-b25a-f87840c059e6" containerName="online" Sep 15 07:12:30.994022 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.993592 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0fd9d10-888c-46c4-a897-80df9e3f4197" containerName="registry" Sep 15 07:12:30.996734 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.996714 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kgbcs/must-gather-wv4lh" Sep 15 07:12:30.999091 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.999068 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kgbcs\"/\"kube-root-ca.crt\"" Sep 15 07:12:30.999091 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.999082 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kgbcs\"/\"openshift-service-ca.crt\"" Sep 15 07:12:30.999259 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:30.999168 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kgbcs\"/\"default-dockercfg-89p9m\"" Sep 15 07:12:31.006547 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.006522 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kgbcs/must-gather-wv4lh"] Sep 15 07:12:31.013106 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.013085 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:31.056435 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.056402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gldzr\" (UniqueName: \"kubernetes.io/projected/0f9667e9-b17a-43a4-9c62-2c6a49248f01-kube-api-access-gldzr\") pod \"must-gather-wv4lh\" (UID: \"0f9667e9-b17a-43a4-9c62-2c6a49248f01\") " pod="openshift-must-gather-kgbcs/must-gather-wv4lh" Sep 15 07:12:31.056573 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.056445 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0f9667e9-b17a-43a4-9c62-2c6a49248f01-must-gather-output\") pod \"must-gather-wv4lh\" (UID: \"0f9667e9-b17a-43a4-9c62-2c6a49248f01\") " pod="openshift-must-gather-kgbcs/must-gather-wv4lh" Sep 15 07:12:31.068018 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.067993 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:31.157796 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.157713 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gldzr\" (UniqueName: \"kubernetes.io/projected/0f9667e9-b17a-43a4-9c62-2c6a49248f01-kube-api-access-gldzr\") pod \"must-gather-wv4lh\" (UID: \"0f9667e9-b17a-43a4-9c62-2c6a49248f01\") " pod="openshift-must-gather-kgbcs/must-gather-wv4lh" Sep 15 07:12:31.157796 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.157762 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0f9667e9-b17a-43a4-9c62-2c6a49248f01-must-gather-output\") pod \"must-gather-wv4lh\" (UID: \"0f9667e9-b17a-43a4-9c62-2c6a49248f01\") " pod="openshift-must-gather-kgbcs/must-gather-wv4lh" Sep 15 07:12:31.158131 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.158113 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0f9667e9-b17a-43a4-9c62-2c6a49248f01-must-gather-output\") pod \"must-gather-wv4lh\" (UID: \"0f9667e9-b17a-43a4-9c62-2c6a49248f01\") " pod="openshift-must-gather-kgbcs/must-gather-wv4lh" Sep 15 07:12:31.165461 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.165444 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gldzr\" (UniqueName: \"kubernetes.io/projected/0f9667e9-b17a-43a4-9c62-2c6a49248f01-kube-api-access-gldzr\") pod \"must-gather-wv4lh\" (UID: \"0f9667e9-b17a-43a4-9c62-2c6a49248f01\") " pod="openshift-must-gather-kgbcs/must-gather-wv4lh" Sep 15 07:12:31.305950 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.305912 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kgbcs/must-gather-wv4lh" Sep 15 07:12:31.572448 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.572426 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kgbcs/must-gather-wv4lh"] Sep 15 07:12:31.574418 ip-10-0-128-50 kubenswrapper[2581]: W0915 07:12:31.574388 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f9667e9_b17a_43a4_9c62_2c6a49248f01.slice/crio-aee54ff985342f2bd6ae5a62ea4fa13b1cebae4c18ed6133d978ea7094b57f3e WatchSource:0}: Error finding container aee54ff985342f2bd6ae5a62ea4fa13b1cebae4c18ed6133d978ea7094b57f3e: Status 404 returned error can't find the container with id aee54ff985342f2bd6ae5a62ea4fa13b1cebae4c18ed6133d978ea7094b57f3e Sep 15 07:12:31.576392 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:31.576372 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:12:32.112547 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:32.112513 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kgbcs/must-gather-wv4lh" event={"ID":"0f9667e9-b17a-43a4-9c62-2c6a49248f01","Type":"ContainerStarted","Data":"aee54ff985342f2bd6ae5a62ea4fa13b1cebae4c18ed6133d978ea7094b57f3e"} Sep 15 07:12:33.119804 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:33.119765 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kgbcs/must-gather-wv4lh" event={"ID":"0f9667e9-b17a-43a4-9c62-2c6a49248f01","Type":"ContainerStarted","Data":"b6f4da5291fa9a5bdc7d14f4f10738ccbcacf05019e7a6c1e8fb4120290c51ae"} Sep 15 07:12:33.120263 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:33.119821 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kgbcs/must-gather-wv4lh" event={"ID":"0f9667e9-b17a-43a4-9c62-2c6a49248f01","Type":"ContainerStarted","Data":"f2341691d1fe7134c2d7201f95c07e7837a3949595cc15595f509ad55fc46de0"} Sep 15 07:12:33.135539 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:33.135485 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kgbcs/must-gather-wv4lh" podStartSLOduration=2.23123511 podStartE2EDuration="3.135469491s" totalImagesPullingTime="904.234381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:12:33.133928378 +0000 UTC m=+2161.208305930" watchObservedRunningTime="2026-09-15 07:12:33.135469491 +0000 UTC m=+2161.209847093" Sep 15 07:12:33.878747 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:33.878717 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s4bpp_3eadb88e-afda-41c4-836a-c044d4a2628d/global-pull-secret-syncer/0.log" Sep 15 07:12:33.922194 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:33.922162 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" Sep 15 07:12:33.994570 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:33.994540 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-50.ec2.internal_cc9a0b5b683b9efc648017b7df9cf169/haproxy/0.log" Sep 15 07:12:36.494832 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:36.494791 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:36.558842 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:36.558799 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-twhfn_c30094d0-3eb2-4349-b7c0-5f33d52d9e84/konnectivity-agent/0.log" limit=1024 Sep 15 07:12:37.561445 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:37.561417 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a2428de1-c6e9-46f3-8f04-970da01d88d2/alertmanager/0.log" Sep 15 07:12:37.588595 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:37.588567 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a2428de1-c6e9-46f3-8f04-970da01d88d2/config-reloader/0.log" Sep 15 07:12:37.622787 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:37.622756 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a2428de1-c6e9-46f3-8f04-970da01d88d2/kube-rbac-proxy-web/0.log" Sep 15 07:12:37.647893 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:37.647830 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a2428de1-c6e9-46f3-8f04-970da01d88d2/kube-rbac-proxy/0.log" Sep 15 07:12:37.674509 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:37.674472 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a2428de1-c6e9-46f3-8f04-970da01d88d2/kube-rbac-proxy-metric/0.log" Sep 15 07:12:37.699234 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:37.699208 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a2428de1-c6e9-46f3-8f04-970da01d88d2/prom-label-proxy/0.log" Sep 15 07:12:37.722901 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:37.722859 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a2428de1-c6e9-46f3-8f04-970da01d88d2/init-config-reloader/0.log" Sep 15 07:12:37.881973 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:37.881879 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-559d484c58-c2mgc_537f65bf-b80e-4e46-8270-131d908e76bd/metrics-server/0.log" Sep 15 07:12:38.035841 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.035815 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k8gk_bd77cbce-b2ff-48d9-a6c1-f41d3030622a/node-exporter/0.log" Sep 15 07:12:38.059389 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.059344 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k8gk_bd77cbce-b2ff-48d9-a6c1-f41d3030622a/kube-rbac-proxy/0.log" Sep 15 07:12:38.082084 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.082059 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k8gk_bd77cbce-b2ff-48d9-a6c1-f41d3030622a/init-textfile/0.log" Sep 15 07:12:38.190160 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.190058 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-pz92z_9615db49-f289-46e6-8f6d-6924c407ba6c/kube-rbac-proxy-main/0.log" Sep 15 07:12:38.212872 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.212842 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-pz92z_9615db49-f289-46e6-8f6d-6924c407ba6c/kube-rbac-proxy-self/0.log" Sep 15 07:12:38.238282 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.238248 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-pz92z_9615db49-f289-46e6-8f6d-6924c407ba6c/openshift-state-metrics/0.log" Sep 15 07:12:38.500682 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.500652 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-gl5lg_72589117-044c-48cd-8d2c-c6a82a587722/prometheus-operator-admission-webhook/0.log" Sep 15 07:12:38.530574 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.530548 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-548cbcfcc8-gj44b_bbbf190f-82bd-41f4-8489-a8c86093f857/telemeter-client/0.log" Sep 15 07:12:38.552952 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.552917 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-548cbcfcc8-gj44b_bbbf190f-82bd-41f4-8489-a8c86093f857/reload/0.log" Sep 15 07:12:38.578302 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.578274 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-548cbcfcc8-gj44b_bbbf190f-82bd-41f4-8489-a8c86093f857/kube-rbac-proxy/0.log" Sep 15 07:12:38.610535 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.610508 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67d6476fb-8vnwr_1d726025-6fe8-44f7-87de-a54afde16e3e/thanos-query/0.log" Sep 15 07:12:38.636620 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.636587 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67d6476fb-8vnwr_1d726025-6fe8-44f7-87de-a54afde16e3e/kube-rbac-proxy-web/0.log" Sep 15 07:12:38.658332 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.658303 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67d6476fb-8vnwr_1d726025-6fe8-44f7-87de-a54afde16e3e/kube-rbac-proxy/0.log" Sep 15 07:12:38.678456 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.678428 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67d6476fb-8vnwr_1d726025-6fe8-44f7-87de-a54afde16e3e/prom-label-proxy/0.log" Sep 15 07:12:38.698793 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.698765 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67d6476fb-8vnwr_1d726025-6fe8-44f7-87de-a54afde16e3e/kube-rbac-proxy-rules/0.log" Sep 15 07:12:38.720645 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:38.720615 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67d6476fb-8vnwr_1d726025-6fe8-44f7-87de-a54afde16e3e/kube-rbac-proxy-metrics/0.log" Sep 15 07:12:40.017063 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:40.017026 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-2kg2n_136a7ac9-282f-44fd-bd26-7284c85bb5df/networking-console-plugin/0.log" Sep 15 07:12:40.860568 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:40.860537 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5bff9b8dcb-v2c9w_e9fa5342-06b9-4d5f-8761-3335c50720d7/console/0.log" Sep 15 07:12:41.466322 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.466284 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj"] Sep 15 07:12:41.472030 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.472006 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.480274 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.480246 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj"] Sep 15 07:12:41.550460 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.550426 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-podres\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.550792 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.550763 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-proc\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.550896 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.550804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-sys\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.550954 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.550901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmnl9\" (UniqueName: \"kubernetes.io/projected/e7da67f6-eada-4a7d-a829-4146b906772c-kube-api-access-nmnl9\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.551018 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.551000 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-lib-modules\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.652417 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652385 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-proc\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.652417 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652419 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-sys\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.652676 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652447 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmnl9\" (UniqueName: \"kubernetes.io/projected/e7da67f6-eada-4a7d-a829-4146b906772c-kube-api-access-nmnl9\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.652676 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-lib-modules\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.652676 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652515 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-proc\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.652676 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652534 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-podres\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.652676 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652515 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-sys\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.652676 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652659 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-podres\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.653013 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.652689 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7da67f6-eada-4a7d-a829-4146b906772c-lib-modules\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.661405 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.661382 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmnl9\" (UniqueName: \"kubernetes.io/projected/e7da67f6-eada-4a7d-a829-4146b906772c-kube-api-access-nmnl9\") pod \"perf-node-gather-daemonset-tdsvj\" (UID: \"e7da67f6-eada-4a7d-a829-4146b906772c\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:41.784861 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:41.784827 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:42.066099 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.066075 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj"] Sep 15 07:12:42.068210 ip-10-0-128-50 kubenswrapper[2581]: W0915 07:12:42.068182 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode7da67f6_eada_4a7d_a829_4146b906772c.slice/crio-c6a8d25b3d7c8fa5e664fe59182e95d0347f3c187262ee3543f2336e5a1ab853 WatchSource:0}: Error finding container c6a8d25b3d7c8fa5e664fe59182e95d0347f3c187262ee3543f2336e5a1ab853: Status 404 returned error can't find the container with id c6a8d25b3d7c8fa5e664fe59182e95d0347f3c187262ee3543f2336e5a1ab853 Sep 15 07:12:42.175882 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.175848 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" event={"ID":"e7da67f6-eada-4a7d-a829-4146b906772c","Type":"ContainerStarted","Data":"ccd15a4fe9524ebc497c9622ba8e98e6c415d70c8b2a8d4f0df723f6de84818b"} Sep 15 07:12:42.176126 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.175902 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" event={"ID":"e7da67f6-eada-4a7d-a829-4146b906772c","Type":"ContainerStarted","Data":"c6a8d25b3d7c8fa5e664fe59182e95d0347f3c187262ee3543f2336e5a1ab853"} Sep 15 07:12:42.176126 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.175977 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:42.194296 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.194256 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" podStartSLOduration=1.194240873 podStartE2EDuration="1.194240873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:12:42.193007227 +0000 UTC m=+2170.267384778" watchObservedRunningTime="2026-09-15 07:12:42.194240873 +0000 UTC m=+2170.268618423" Sep 15 07:12:42.219274 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.219253 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qhvtn_2a563a2a-f128-4058-b119-fe1d206e19fc/dns/0.log" Sep 15 07:12:42.237001 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.236982 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qhvtn_2a563a2a-f128-4058-b119-fe1d206e19fc/kube-rbac-proxy/0.log" Sep 15 07:12:42.301055 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.301031 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mx7sh_c148b74f-c481-4720-bf3d-50bd390af2da/dns-node-resolver/0.log" Sep 15 07:12:42.755216 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:42.755168 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tjm9m_363050d3-1590-4804-b6cc-48cecf994d16/node-ca/0.log" Sep 15 07:12:43.723453 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:43.723417 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v5vjb_1c787e62-1891-409b-890d-90dc4095665b/serve-healthcheck-canary/0.log" Sep 15 07:12:44.143663 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:44.143629 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gdw2k_d25447d6-a2c6-4d8c-b5bc-b684209f6e02/kube-rbac-proxy/0.log" Sep 15 07:12:44.162405 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:44.162381 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gdw2k_d25447d6-a2c6-4d8c-b5bc-b684209f6e02/exporter/0.log" Sep 15 07:12:44.181378 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:44.181356 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gdw2k_d25447d6-a2c6-4d8c-b5bc-b684209f6e02/extractor/0.log" Sep 15 07:12:48.194517 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:48.194470 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-tdsvj" Sep 15 07:12:49.756562 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:49.756532 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n287w_e301ce06-d062-4d82-9bf1-a91f189bb680/kube-multus-additional-cni-plugins/0.log" Sep 15 07:12:49.775392 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:49.775366 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n287w_e301ce06-d062-4d82-9bf1-a91f189bb680/egress-router-binary-copy/0.log" Sep 15 07:12:49.793742 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:49.793712 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n287w_e301ce06-d062-4d82-9bf1-a91f189bb680/cni-plugins/0.log" Sep 15 07:12:49.811598 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:49.811572 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n287w_e301ce06-d062-4d82-9bf1-a91f189bb680/bond-cni-plugin/0.log" Sep 15 07:12:49.830896 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:49.830864 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n287w_e301ce06-d062-4d82-9bf1-a91f189bb680/routeoverride-cni/0.log" Sep 15 07:12:49.849564 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:49.849536 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n287w_e301ce06-d062-4d82-9bf1-a91f189bb680/whereabouts-cni-bincopy/0.log" Sep 15 07:12:49.869305 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:49.869274 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n287w_e301ce06-d062-4d82-9bf1-a91f189bb680/whereabouts-cni/0.log" Sep 15 07:12:49.904252 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:49.904197 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hq4tx_e4fd90f4-f3b4-49c5-9fc0-d6352b18ec96/kube-multus/0.log" Sep 15 07:12:50.059751 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:50.059718 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xm2jl_9e225fca-64e1-4eb9-9424-c54b0b77cad6/network-metrics-daemon/0.log" Sep 15 07:12:50.079346 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:50.079318 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xm2jl_9e225fca-64e1-4eb9-9424-c54b0b77cad6/kube-rbac-proxy/0.log" Sep 15 07:12:50.913726 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:50.913697 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6p2cg_730697a1-01d9-418d-8658-58d762ecdc93/ovn-controller/0.log" Sep 15 07:12:50.929338 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:50.929312 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6p2cg_730697a1-01d9-418d-8658-58d762ecdc93/ovn-acl-logging/0.log" Sep 15 07:12:50.954892 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:50.954860 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6p2cg_730697a1-01d9-418d-8658-58d762ecdc93/kube-rbac-proxy-node/0.log" Sep 15 07:12:50.975460 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:50.975388 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6p2cg_730697a1-01d9-418d-8658-58d762ecdc93/kube-rbac-proxy-ovn-metrics/0.log" Sep 15 07:12:50.990986 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:50.990960 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6p2cg_730697a1-01d9-418d-8658-58d762ecdc93/northd/0.log" Sep 15 07:12:51.015020 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:51.014989 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6p2cg_730697a1-01d9-418d-8658-58d762ecdc93/nbdb/0.log" Sep 15 07:12:51.037745 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:51.037720 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6p2cg_730697a1-01d9-418d-8658-58d762ecdc93/sbdb/0.log" Sep 15 07:12:51.077518 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:51.077488 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6p2cg_730697a1-01d9-418d-8658-58d762ecdc93/ovnkube-controller/0.log" Sep 15 07:12:52.508610 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:52.508581 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-pwmjh_5ed75ee0-83e9-4fe4-8335-43aba5fcba49/network-check-target-container/0.log" Sep 15 07:12:53.350448 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:53.350420 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-p8pr4_4ac014bb-37f9-44d9-a7b5-fe7a9ae0fae2/iptables-alerter/0.log" Sep 15 07:12:54.032743 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:54.032712 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-vvxb9_8f9c11b3-237a-4501-afcc-8f4c8e30a327/tuned/0.log" Sep 15 07:12:56.837615 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:56.837587 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-568869fdc5-twpgb_ce8dfd64-bb78-4c87-86b7-ba526cf372e5/service-ca-controller/0.log" Sep 15 07:12:57.240717 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:57.240635 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-dmgkf_71955717-dffc-4ec7-8472-c320b24abff9/csi-driver/0.log" Sep 15 07:12:57.261888 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:57.261861 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-dmgkf_71955717-dffc-4ec7-8472-c320b24abff9/csi-node-driver-registrar/0.log" Sep 15 07:12:57.279669 ip-10-0-128-50 kubenswrapper[2581]: I0915 07:12:57.279642 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-dmgkf_71955717-dffc-4ec7-8472-c320b24abff9/csi-liveness-probe/0.log"