Sep 09 08:37:12.478278 ip-10-0-133-59 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 08:37:12.478292 ip-10-0-133-59 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 08:37:12.478302 ip-10-0-133-59 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 08:37:12.478519 ip-10-0-133-59 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 08:37:23.948776 ip-10-0-133-59 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 08:37:23.948792 ip-10-0-133-59 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 049cdb6947dd48d8a7bbbadddb1faa57 -- Sep 09 08:39:16.058263 ip-10-0-133-59 systemd[1]: Starting Kubernetes Kubelet... Sep 09 08:39:16.420612 ip-10-0-133-59 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 08:39:16.420612 ip-10-0-133-59 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 08:39:16.420612 ip-10-0-133-59 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 08:39:16.420612 ip-10-0-133-59 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 08:39:16.420612 ip-10-0-133-59 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 08:39:16.422534 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.422453 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 08:39:16.426041 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426026 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 08:39:16.426041 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426041 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426046 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426049 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426053 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426056 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426059 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426062 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426064 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426067 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426070 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426072 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426075 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426078 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426080 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426083 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426093 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426096 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426098 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426101 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426104 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 08:39:16.426105 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426107 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426110 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426113 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426116 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426119 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426121 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426124 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426127 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426129 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426131 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426135 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426139 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426142 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426145 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426147 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426150 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426152 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426154 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426157 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 08:39:16.426578 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426160 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426162 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426165 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426168 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426170 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426173 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426175 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426178 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426180 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426182 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426185 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426188 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426190 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426193 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426196 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426199 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426201 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426204 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426208 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426212 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 08:39:16.427054 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426215 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426218 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426221 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426223 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426226 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426229 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426232 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426235 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426237 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426240 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426243 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426246 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426249 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426251 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426254 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426256 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426259 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426261 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426264 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426266 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 08:39:16.427573 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426269 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426271 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426274 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426276 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426279 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426282 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426285 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426288 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426291 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426294 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426296 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426299 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426301 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426304 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426306 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426309 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426313 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426316 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 08:39:16.428088 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.426318 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428334 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428341 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428345 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428348 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428351 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428354 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428357 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428360 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428363 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428365 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428368 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428370 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428373 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428375 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428378 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428381 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428383 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428386 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428389 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 08:39:16.428524 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428391 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428394 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428396 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428399 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428401 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428404 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428407 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428409 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428412 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428414 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428417 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428419 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428422 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428424 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428428 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428431 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428433 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428436 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428438 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428442 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 08:39:16.429038 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428444 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428447 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428449 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428452 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428454 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428457 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428459 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428462 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428464 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428467 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428470 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428472 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428475 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428477 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428482 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428486 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428489 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428491 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428494 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 08:39:16.429527 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428497 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428499 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428502 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428504 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428507 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428509 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428512 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428517 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428520 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428523 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428526 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428528 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428531 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428534 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428539 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428542 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428545 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428548 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428551 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 08:39:16.429996 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428553 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428556 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428559 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428562 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428573 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428578 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428581 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428584 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428587 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428589 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428592 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428595 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428597 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428600 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428603 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428605 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428608 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428610 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428613 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428615 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 08:39:16.430463 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428619 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.428622 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429041 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429048 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429051 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429054 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429058 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429061 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429064 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429066 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429069 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429072 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429074 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429077 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429079 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429082 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429085 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429089 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429093 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429096 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 08:39:16.430946 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429099 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429101 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429104 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429107 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429110 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429113 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429116 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429118 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429121 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429123 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429126 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429128 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429132 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429136 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429139 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429141 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429144 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429146 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429150 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 08:39:16.431458 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429154 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429157 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429160 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429163 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429165 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429168 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429171 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429174 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429176 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429179 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429181 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429185 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429187 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429189 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429192 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429194 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429197 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429199 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429202 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429204 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 08:39:16.431921 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429207 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429209 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429212 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429214 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429223 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429226 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429228 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429231 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429234 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429236 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429239 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429242 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429244 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429247 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429250 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429253 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429256 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429258 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429261 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 08:39:16.432472 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429264 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429266 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429269 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429271 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429274 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429276 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429279 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429281 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429285 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429288 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429290 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429293 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429296 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429298 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429300 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429303 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429305 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429308 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429312 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429314 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 08:39:16.432936 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429317 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429320 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.429322 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429395 2577 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429435 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429443 2577 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429447 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429451 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429455 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429459 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429462 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429466 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429469 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429472 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429476 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429479 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429482 2577 flags.go:64] FLAG: --cgroup-root="" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429484 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429488 2577 flags.go:64] FLAG: --client-ca-file="" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429491 2577 flags.go:64] FLAG: --cloud-provider="external" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429494 2577 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429498 2577 flags.go:64] FLAG: --cluster-domain="" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429500 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 08:39:16.433442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429503 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429506 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429510 2577 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429513 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429516 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429519 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429522 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429528 2577 flags.go:64] FLAG: --contention-profiling="false" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429531 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429534 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429537 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429540 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429544 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429547 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429550 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429553 2577 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429556 2577 flags.go:64] FLAG: --enable-server="true" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429559 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429564 2577 flags.go:64] FLAG: --event-burst="100" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429567 2577 flags.go:64] FLAG: --event-qps="50" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429570 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429573 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429576 2577 flags.go:64] FLAG: --eviction-hard="" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429580 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 08:39:16.434025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429583 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429585 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429588 2577 flags.go:64] FLAG: --eviction-soft="" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429591 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429594 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429597 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429600 2577 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429602 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429605 2577 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429608 2577 flags.go:64] FLAG: --feature-gates="" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429612 2577 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429614 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429617 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429622 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429625 2577 flags.go:64] FLAG: --healthz-port="10248" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429629 2577 flags.go:64] FLAG: --help="false" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429632 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429635 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429639 2577 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429642 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429645 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429648 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429652 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 08:39:16.434610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429654 2577 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429657 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429661 2577 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429664 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429668 2577 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429671 2577 flags.go:64] FLAG: --kube-reserved="" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429673 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429676 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429679 2577 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429682 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429685 2577 flags.go:64] FLAG: --lock-file="" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429688 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429691 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429694 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429699 2577 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429702 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429704 2577 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429707 2577 flags.go:64] FLAG: --logging-format="text" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429710 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429714 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429716 2577 flags.go:64] FLAG: --manifest-url="" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429719 2577 flags.go:64] FLAG: --manifest-url-header="" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429724 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429727 2577 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429732 2577 flags.go:64] FLAG: --max-pods="110" Sep 09 08:39:16.435271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429736 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429739 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429743 2577 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429745 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429748 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429751 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429755 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429761 2577 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429764 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429767 2577 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429770 2577 flags.go:64] FLAG: --pod-cidr="" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429774 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429779 2577 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429782 2577 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429785 2577 flags.go:64] FLAG: --pods-per-core="0" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429788 2577 flags.go:64] FLAG: --port="10250" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429791 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429794 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-079e6a5fe75b2605d" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429797 2577 flags.go:64] FLAG: --qos-reserved="" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429800 2577 flags.go:64] FLAG: --read-only-port="10255" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429803 2577 flags.go:64] FLAG: --register-node="true" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429805 2577 flags.go:64] FLAG: --register-with-taints="" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429809 2577 flags.go:64] FLAG: --registry-burst="10" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429812 2577 flags.go:64] FLAG: --registry-qps="5" Sep 09 08:39:16.435875 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429814 2577 flags.go:64] FLAG: --reserved-cpus="" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429817 2577 flags.go:64] FLAG: --reserved-memory="" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429820 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429823 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429826 2577 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429829 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429833 2577 flags.go:64] FLAG: --runonce="false" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429838 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429841 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429844 2577 flags.go:64] FLAG: --seccomp-default="false" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429847 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429850 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429853 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429856 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429860 2577 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429862 2577 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429865 2577 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429868 2577 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429871 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429874 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429877 2577 flags.go:64] FLAG: --system-cgroups="" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429881 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429886 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429888 2577 flags.go:64] FLAG: --tls-cert-file="" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429891 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 08:39:16.436542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429895 2577 flags.go:64] FLAG: --tls-min-version="" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429898 2577 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429901 2577 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429903 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429906 2577 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429909 2577 flags.go:64] FLAG: --v="2" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429913 2577 flags.go:64] FLAG: --version="false" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429916 2577 flags.go:64] FLAG: --vmodule="" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429920 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.429923 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430056 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430061 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430064 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430068 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430072 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430075 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430078 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430081 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430084 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430086 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430089 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430092 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 08:39:16.437179 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430095 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430099 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430101 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430104 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430107 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430110 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430112 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430115 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430118 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430120 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430123 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430132 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430135 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430137 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430140 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430142 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430145 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430148 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430151 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 08:39:16.437702 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430153 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430156 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430159 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430162 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430166 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430168 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430171 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430175 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430179 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430182 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430185 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430188 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430191 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430194 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430196 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430199 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430202 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430204 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430207 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 08:39:16.438174 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430210 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430213 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430215 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430218 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430220 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430224 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430226 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430229 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430231 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430234 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430236 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430239 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430242 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430246 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430248 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430251 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430253 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430257 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430260 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430263 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 08:39:16.438654 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430265 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430268 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430270 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430273 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430276 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430278 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430281 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430283 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430286 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430289 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430291 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430294 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430296 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430299 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430302 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430305 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430307 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430310 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430312 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 08:39:16.439153 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430316 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430320 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430323 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430325 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430328 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430330 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430333 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430335 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430338 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.430340 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.430851 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.437784 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.437797 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.437815 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.437822 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 08:39:16.439666 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437892 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437898 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437901 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437904 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437907 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437910 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437912 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437915 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437918 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437921 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437923 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437926 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437929 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437932 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437934 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437937 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437940 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437943 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437946 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437949 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 08:39:16.440065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437951 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437954 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437957 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437959 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437962 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437964 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437968 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437987 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437991 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437995 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.437998 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438001 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438003 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438006 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438009 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438012 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438014 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438017 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438019 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 08:39:16.440556 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438022 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438025 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438027 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438030 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438033 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438035 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438038 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438041 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438043 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438046 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438049 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438051 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438054 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438056 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438060 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438064 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438066 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438069 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438071 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 08:39:16.441049 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438074 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438076 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438079 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438082 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438085 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438087 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438090 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438093 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438096 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438099 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438101 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438104 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438106 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438109 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438112 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438116 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438119 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438122 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438125 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 08:39:16.441508 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438127 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438131 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438133 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438136 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438139 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438141 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438143 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438146 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438148 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438151 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438154 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438156 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438159 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438161 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438164 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438166 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438169 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438172 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438174 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438177 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 08:39:16.442008 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438180 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438183 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.438188 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438314 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438319 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438322 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438325 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438328 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438331 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438333 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438336 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438340 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438343 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438347 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 08:39:16.442525 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438351 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438354 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438357 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438360 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438363 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438366 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438368 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438371 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438373 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438376 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438378 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438381 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438383 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438386 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438388 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438392 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438394 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438397 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438399 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438402 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 08:39:16.442869 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438404 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438407 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438409 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438412 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438414 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438417 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438420 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438422 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438425 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438429 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438432 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438435 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438437 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438441 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438443 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438446 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438448 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438451 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438453 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438456 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 08:39:16.443459 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438458 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438461 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438463 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438466 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438468 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438471 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438474 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438477 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438479 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438481 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438484 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438486 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438489 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438491 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438494 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438496 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438499 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438501 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438503 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 08:39:16.443960 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438506 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438508 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438511 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438513 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438515 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438518 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438520 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438523 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438526 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438528 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438531 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438533 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438535 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438538 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438541 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438543 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438545 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438548 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438550 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438553 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 08:39:16.444449 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438555 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438558 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438561 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438563 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438566 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438568 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438571 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438574 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:16.438576 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.438581 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.438671 2577 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.441883 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.442935 2577 server.go:1019] "Starting client certificate rotation" Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.443031 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 08:39:16.444935 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.443068 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 08:39:16.445356 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.445288 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 08:39:16.446575 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.446558 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 08:39:16.456530 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.456512 2577 log.go:25] "Validated CRI v1 runtime API" Sep 09 08:39:16.459189 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.459172 2577 log.go:25] "Validated CRI v1 image API" Sep 09 08:39:16.460620 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.460593 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 08:39:16.466422 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.466401 2577 fs.go:135] Filesystem UUIDs: map[5ad83c03-bdd5-4d4a-832f-4e393e554bea:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 df38255d-bb5a-4e87-9c15-cfa5b8006a59:/dev/nvme0n1p3] Sep 09 08:39:16.466493 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.466421 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 08:39:16.468200 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.468182 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 08:39:16.471434 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.471326 2577 manager.go:216] Machine: {Timestamp:2026-09-09 08:39:16.470106563 +0000 UTC m=+0.303470132 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3121964 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21f525b5ba1ee78e0ac1d2843953ea SystemUUID:ec21f525-b5ba-1ee7-8e0a-c1d2843953ea BootID:049cdb69-47dd-48d8-a7bb-badddb1faa57 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d2:27:49:f4:49 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d2:27:49:f4:49 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:9d:3d:2a:4f:e0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 08:39:16.471434 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.471431 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 08:39:16.471549 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.471543 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 08:39:16.474231 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.474209 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 08:39:16.474370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.474233 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-59.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 08:39:16.474855 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.474846 2577 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 08:39:16.474885 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.474858 2577 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 08:39:16.474885 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.474865 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 08:39:16.475479 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.475470 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 08:39:16.476642 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.476632 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 08:39:16.480705 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.480693 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 09 08:39:16.480826 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.480816 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 08:39:16.480857 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.480847 2577 kubelet.go:486] "Attempting to sync node with API server" Sep 09 08:39:16.480892 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.480858 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 08:39:16.480892 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.480872 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 08:39:16.480892 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.480882 2577 kubelet.go:398] "Adding apiserver pod source" Sep 09 08:39:16.480892 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.480890 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 08:39:16.481264 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.481252 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 08:39:16.481303 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.481273 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 08:39:16.483798 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.483776 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 08:39:16.487391 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.487374 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 08:39:16.487453 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.487405 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 08:39:16.487569 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.487557 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 08:39:16.487622 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.487571 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 08:39:16.487622 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.487577 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 08:39:16.487622 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.487584 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 08:39:16.487622 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.487592 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 08:39:16.487622 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.487597 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 08:39:16.488012 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.488000 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 08:39:16.488012 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.488010 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 08:39:16.488130 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.488019 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 08:39:16.488130 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.488026 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 08:39:16.488130 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.488038 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 08:39:16.488130 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.488050 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 08:39:16.489012 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.489000 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 08:39:16.489012 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.489011 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 08:39:16.490302 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.490280 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 08:39:16.490302 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.490280 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 08:39:16.492736 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.492723 2577 server.go:1268] "Started kubelet" Sep 09 08:39:16.492825 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.492805 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 08:39:16.492888 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.492807 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 08:39:16.492888 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.492876 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 08:39:16.493555 ip-10-0-133-59 systemd[1]: Started Kubernetes Kubelet. Sep 09 08:39:16.494043 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.494025 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 08:39:16.497934 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.497908 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dhv8q" Sep 09 08:39:16.498661 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.498645 2577 server.go:310] "Adding debug handlers to kubelet server" Sep 09 08:39:16.500106 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.500082 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 08:39:16.504599 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.504577 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dhv8q" Sep 09 08:39:16.505178 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.505158 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 08:39:16.505270 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.505179 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 08:39:16.505323 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.505284 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:16.505700 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.505685 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 08:39:16.505786 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.505704 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 08:39:16.505786 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.505685 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 08:39:16.505895 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.505833 2577 reconstruct.go:97] "Volume reconstruction finished" Sep 09 08:39:16.505895 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.505841 2577 reconciler.go:29] "Reconciler: start to sync state" Sep 09 08:39:16.507275 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.507257 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 08:39:16.507275 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.507274 2577 factory.go:55] Registering systemd factory Sep 09 08:39:16.507400 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.507281 2577 factory.go:223] Registration of the systemd container factory successfully Sep 09 08:39:16.507815 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.507778 2577 factory.go:153] Registering CRI-O factory Sep 09 08:39:16.507815 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.507809 2577 factory.go:223] Registration of the crio container factory successfully Sep 09 08:39:16.507960 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.507862 2577 factory.go:103] Registering Raw factory Sep 09 08:39:16.507960 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.507879 2577 manager.go:1196] Started watching for new ooms in manager Sep 09 08:39:16.507960 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.507942 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-59.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 08:39:16.508163 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.508137 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 08:39:16.508301 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.508222 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 08:39:16.508384 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.508365 2577 manager.go:349] Starting recovery of all containers Sep 09 08:39:16.508741 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.507916 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-59.ec2.internal.18d39a52961cf009 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-59.ec2.internal,UID:ip-10-0-133-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-59.ec2.internal,},FirstTimestamp:2026-09-09 08:39:16.492693513 +0000 UTC m=+0.326057063,LastTimestamp:2026-09-09 08:39:16.492693513 +0000 UTC m=+0.326057063,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-59.ec2.internal,}" Sep 09 08:39:16.517403 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.517379 2577 manager.go:354] Recovery completed Sep 09 08:39:16.522171 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.522156 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 08:39:16.524273 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.524256 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientMemory" Sep 09 08:39:16.524351 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.524283 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 08:39:16.524351 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.524293 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientPID" Sep 09 08:39:16.524819 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.524804 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 08:39:16.524819 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.524816 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 08:39:16.524930 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.524834 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 09 08:39:16.524930 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.524853 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 08:39:16.525584 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.525569 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientMemory" Sep 09 08:39:16.525645 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.525599 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 08:39:16.525645 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.525612 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientPID" Sep 09 08:39:16.526177 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.526119 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-59.ec2.internal.18d39a5297fec2cd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-59.ec2.internal,UID:ip-10-0-133-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-59.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-59.ec2.internal,},FirstTimestamp:2026-09-09 08:39:16.524270285 +0000 UTC m=+0.357633837,LastTimestamp:2026-09-09 08:39:16.524270285 +0000 UTC m=+0.357633837,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-59.ec2.internal,}" Sep 09 08:39:16.526807 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.526796 2577 policy_none.go:49] "None policy: Start" Sep 09 08:39:16.526849 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.526810 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 08:39:16.526849 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.526820 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 08:39:16.528892 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.528880 2577 policy_none.go:47] "Start" Sep 09 08:39:16.554660 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.554644 2577 manager.go:337] "Starting Device Plugin manager" Sep 09 08:39:16.554735 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.554680 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 08:39:16.554735 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.554690 2577 server.go:87] "Starting device plugin registration server" Sep 09 08:39:16.554960 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.554943 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 08:39:16.555115 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.554960 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 08:39:16.555207 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.555135 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 08:39:16.557110 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.555570 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 08:39:16.557110 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.555587 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 08:39:16.557110 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.556267 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 08:39:16.557110 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.556322 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:16.626505 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.626473 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 08:39:16.627688 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.627660 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 08:39:16.627779 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.627705 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 08:39:16.627779 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.627727 2577 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 08:39:16.627779 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.627764 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 08:39:16.630289 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.630268 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 08:39:16.655960 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.655945 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 08:39:16.656782 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.656768 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientMemory" Sep 09 08:39:16.656858 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.656797 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 08:39:16.656858 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.656808 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientPID" Sep 09 08:39:16.656858 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.656833 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.662703 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.662689 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.662761 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.662710 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-59.ec2.internal\": node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:16.676272 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.676223 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:16.728035 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.728013 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal"] Sep 09 08:39:16.728122 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.728077 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 08:39:16.729944 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.729929 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientMemory" Sep 09 08:39:16.730016 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.729958 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 08:39:16.730016 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.729969 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientPID" Sep 09 08:39:16.732206 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.732195 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 08:39:16.732329 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.732315 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.732376 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.732346 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 08:39:16.733017 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.733003 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientMemory" Sep 09 08:39:16.733089 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.733028 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 08:39:16.733089 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.733039 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientPID" Sep 09 08:39:16.733089 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.733043 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientMemory" Sep 09 08:39:16.733089 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.733065 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 08:39:16.733089 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.733074 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientPID" Sep 09 08:39:16.735186 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.735167 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.735249 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.735193 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 08:39:16.735858 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.735839 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientMemory" Sep 09 08:39:16.735937 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.735867 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 08:39:16.735937 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.735878 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeHasSufficientPID" Sep 09 08:39:16.764658 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.764634 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-59.ec2.internal\" not found" node="ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.769058 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.769045 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-59.ec2.internal\" not found" node="ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.777082 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.777068 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:16.807818 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.807795 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50583baa71de9d4d5e83da0f276b22a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal\" (UID: \"50583baa71de9d4d5e83da0f276b22a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.807899 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.807820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50583baa71de9d4d5e83da0f276b22a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal\" (UID: \"50583baa71de9d4d5e83da0f276b22a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.807899 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.807837 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/581b21db7cc54e6fc2cafbc222baf6c8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-59.ec2.internal\" (UID: \"581b21db7cc54e6fc2cafbc222baf6c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.877604 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.877584 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:16.907998 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.907957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/581b21db7cc54e6fc2cafbc222baf6c8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-59.ec2.internal\" (UID: \"581b21db7cc54e6fc2cafbc222baf6c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.908100 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.908004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50583baa71de9d4d5e83da0f276b22a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal\" (UID: \"50583baa71de9d4d5e83da0f276b22a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.908100 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.908021 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50583baa71de9d4d5e83da0f276b22a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal\" (UID: \"50583baa71de9d4d5e83da0f276b22a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.908100 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.908049 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/581b21db7cc54e6fc2cafbc222baf6c8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-59.ec2.internal\" (UID: \"581b21db7cc54e6fc2cafbc222baf6c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.908100 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.908070 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50583baa71de9d4d5e83da0f276b22a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal\" (UID: \"50583baa71de9d4d5e83da0f276b22a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.908100 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:16.908049 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50583baa71de9d4d5e83da0f276b22a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal\" (UID: \"50583baa71de9d4d5e83da0f276b22a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:16.978257 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:16.978184 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.067832 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.067798 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:17.072820 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.072614 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" Sep 09 08:39:17.079179 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.079158 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.179640 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.179612 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.280446 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.280374 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.384105 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.384052 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.444035 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.443995 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 08:39:17.444476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.444166 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 08:39:17.484416 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.484394 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.508715 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.508678 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 08:34:17 +0000 UTC" deadline="2028-02-25 16:27:45.829027753 +0000 UTC" Sep 09 08:39:17.508715 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.508709 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12823h48m28.320322692s" Sep 09 08:39:17.585513 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.585438 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.609367 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.609345 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 08:39:17.685948 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.685781 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.786606 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.786563 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.803850 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:17.803819 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581b21db7cc54e6fc2cafbc222baf6c8.slice/crio-3f0257ab9f4d4580cc39b5dfd87157e76879c3d4c7554b40107e3aeaf43ce649 WatchSource:0}: Error finding container 3f0257ab9f4d4580cc39b5dfd87157e76879c3d4c7554b40107e3aeaf43ce649: Status 404 returned error can't find the container with id 3f0257ab9f4d4580cc39b5dfd87157e76879c3d4c7554b40107e3aeaf43ce649 Sep 09 08:39:17.811211 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.811194 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 08:39:17.818503 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:17.818481 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50583baa71de9d4d5e83da0f276b22a0.slice/crio-c7667c1b5f87ddf90c5657c2aca75d14764cd1dd906569d0fb954f3f2b5b4a39 WatchSource:0}: Error finding container c7667c1b5f87ddf90c5657c2aca75d14764cd1dd906569d0fb954f3f2b5b4a39: Status 404 returned error can't find the container with id c7667c1b5f87ddf90c5657c2aca75d14764cd1dd906569d0fb954f3f2b5b4a39 Sep 09 08:39:17.887902 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:17.887827 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-59.ec2.internal\" not found" Sep 09 08:39:17.895477 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.895460 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 08:39:17.906116 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.906094 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" Sep 09 08:39:17.917064 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.917046 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 08:39:17.918143 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.918130 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" Sep 09 08:39:17.927648 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:17.927632 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 08:39:18.015178 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.015157 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 08:39:18.273253 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.273170 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 08:39:18.483777 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.483522 2577 apiserver.go:52] "Watching apiserver" Sep 09 08:39:18.492365 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.492069 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 08:39:18.493717 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.493488 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-c25ds","kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6","openshift-cluster-node-tuning-operator/tuned-tvfvr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal","openshift-multus/multus-cqjwk","openshift-network-diagnostics/network-check-target-9w6g4","openshift-network-operator/iptables-alerter-tw4wg","kube-system/konnectivity-agent-9qxkj","openshift-dns/node-resolver-lcbk4","openshift-image-registry/node-ca-jmn4t","openshift-multus/multus-additional-cni-plugins-q76kl","openshift-multus/network-metrics-daemon-wqxmh","openshift-ovn-kubernetes/ovnkube-node-jw7m5"] Sep 09 08:39:18.496311 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.496286 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:18.496422 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.496375 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:18.498638 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.498611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.501525 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.501060 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.501525 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.501389 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 08:39:18.501694 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.501549 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 08:39:18.504443 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.504363 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 08:39:18.504913 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.504892 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 08:39:18.505154 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.505138 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5z6lq\"" Sep 09 08:39:18.505435 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.505421 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 08:39:18.505788 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.505766 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 08:39:18.506782 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.506764 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nkllp\"" Sep 09 08:39:18.507020 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.506998 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 08:39:18.507402 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.507359 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.510332 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.510051 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6zl56\"" Sep 09 08:39:18.510440 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.510336 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 08:39:18.510566 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.510484 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 08:39:18.510566 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.510543 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.513771 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.513343 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.514179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.514160 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zpj5v\"" Sep 09 08:39:18.514548 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.514271 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 08:39:18.514548 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.514301 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 08:39:18.514548 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.514446 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 08:39:18.514548 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.514498 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 08:39:18.514548 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.514509 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 08:39:18.514810 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.514729 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 08:39:18.515328 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:18.515328 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515225 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-device-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.515328 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-host\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.515547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-ovn\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4z7t\" (UniqueName: \"kubernetes.io/projected/333ce92a-033d-45bc-bbfc-78ea69ff251c-kube-api-access-r4z7t\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.515547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515411 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-socket-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.515547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515436 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysctl-conf\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.515547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515459 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-systemd-units\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515485 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-var-lib-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/333ce92a-033d-45bc-bbfc-78ea69ff251c-host\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.515547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515530 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/333ce92a-033d-45bc-bbfc-78ea69ff251c-serviceca\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515554 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-tuned\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515580 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-kubelet\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515604 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-cni-bin\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovnkube-script-lib\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515653 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-registration-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515677 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-lib-modules\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-var-lib-kubelet\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-cni-netd\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovn-node-metrics-cert\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515805 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-modprobe-d\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-systemd\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-run\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515875 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-slash\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8942n\" (UniqueName: \"kubernetes.io/projected/7f562836-0b51-4578-a0d0-df5bba3702c8-kube-api-access-8942n\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.515950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515940 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515951 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-kubernetes\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515955 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dhz45\"" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.515976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l657j\" (UniqueName: \"kubernetes.io/projected/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-kube-api-access-l657j\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516170 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-etc-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516209 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-log-socket\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516241 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516261 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-env-overrides\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516284 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516289 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516313 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysconfig\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-sys\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516358 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-run-netns\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516381 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-systemd\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovnkube-config\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516429 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xmds\" (UniqueName: \"kubernetes.io/projected/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-kube-api-access-7xmds\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516469 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-sys-fs\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.516823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysctl-d\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.517738 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516515 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-tmp\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.517738 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516238 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 08:39:18.517738 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-node-log\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.517738 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.516567 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.517738 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.517088 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:18.519706 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.519684 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:18.519805 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.519767 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:18.521689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.521285 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 08:39:18.521689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.521556 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 08:39:18.521689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.521562 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9mcg9\"" Sep 09 08:39:18.524179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.523854 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 08:39:18.524263 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.524176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.526943 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.526520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.527817 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.527796 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 08:39:18.527916 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.527796 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 08:39:18.528283 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.528106 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mc2wv\"" Sep 09 08:39:18.529087 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.528606 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.529087 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.528668 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:18.529087 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.528909 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 08:39:18.530264 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.530228 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 08:39:18.530701 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.530527 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-slb2t\"" Sep 09 08:39:18.530701 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.530584 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 08:39:18.531492 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.531292 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.534213 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.533885 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 08:39:18.534213 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.533885 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 08:39:18.535931 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.535896 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zwhl4\"" Sep 09 08:39:18.547519 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.547454 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-78bdb" Sep 09 08:39:18.558340 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.558319 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-78bdb" Sep 09 08:39:18.608196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.607872 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617049 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e4e33c67-cd89-4b31-946f-0792b3e9a131-konnectivity-ca\") pod \"konnectivity-agent-9qxkj\" (UID: \"e4e33c67-cd89-4b31-946f-0792b3e9a131\") " pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-hosts-file\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617113 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-tmp-dir\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617127 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-system-cni-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-kubernetes\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l657j\" (UniqueName: \"kubernetes.io/projected/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-kube-api-access-l657j\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617174 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-env-overrides\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-conf-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617231 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-sys\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617251 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-run-netns\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617266 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-systemd\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovnkube-config\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617300 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-cni-multus\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617325 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-etc-kubernetes\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.619109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-tmp\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a228400-e2ea-43dc-85c0-9bf945bb0872-host-slash\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65a71213-3745-4fe0-8fd1-372771761d7a-cni-binary-copy\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617389 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-cni-bin\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65a71213-3745-4fe0-8fd1-372771761d7a-multus-daemon-config\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-device-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617451 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-host\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-ovn\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-k8s-cni-cncf-io\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4z7t\" (UniqueName: \"kubernetes.io/projected/333ce92a-033d-45bc-bbfc-78ea69ff251c-kube-api-access-r4z7t\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617526 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-socket-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-var-lib-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617562 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/333ce92a-033d-45bc-bbfc-78ea69ff251c-host\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617579 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-kubelet\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617603 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-lib-modules\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617623 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-cni-netd\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617639 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-cni-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-modprobe-d\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617690 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-slash\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617705 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc4jf\" (UniqueName: \"kubernetes.io/projected/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-kube-api-access-dc4jf\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn7kq\" (UniqueName: \"kubernetes.io/projected/65a71213-3745-4fe0-8fd1-372771761d7a-kube-api-access-tn7kq\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617737 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8942n\" (UniqueName: \"kubernetes.io/projected/7f562836-0b51-4578-a0d0-df5bba3702c8-kube-api-access-8942n\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617753 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-etc-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-log-socket\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617783 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617799 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cbbp\" (UniqueName: \"kubernetes.io/projected/a4992530-e8b6-4ebd-bc09-04038870dfbf-kube-api-access-5cbbp\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617815 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cnibin\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysconfig\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xmds\" (UniqueName: \"kubernetes.io/projected/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-kube-api-access-7xmds\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6a228400-e2ea-43dc-85c0-9bf945bb0872-iptables-alerter-script\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617890 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:18.620991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-system-cni-dir\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-multus-certs\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617941 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-sys-fs\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysctl-d\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.617981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-node-log\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618006 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e9b855f-9ef8-4c2f-903d-492a4119d176-kubelet-config\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-socket-dir-parent\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618092 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e9b855f-9ef8-4c2f-903d-492a4119d176-dbus\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4h4w\" (UniqueName: \"kubernetes.io/projected/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-kube-api-access-k4h4w\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618134 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysctl-conf\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-systemd-units\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckzj4\" (UniqueName: \"kubernetes.io/projected/6a228400-e2ea-43dc-85c0-9bf945bb0872-kube-api-access-ckzj4\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-netns\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618203 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-hostroot\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/333ce92a-033d-45bc-bbfc-78ea69ff251c-serviceca\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.621789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-tuned\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-cni-bin\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovnkube-script-lib\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618310 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e4e33c67-cd89-4b31-946f-0792b3e9a131-agent-certs\") pod \"konnectivity-agent-9qxkj\" (UID: \"e4e33c67-cd89-4b31-946f-0792b3e9a131\") " pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cni-binary-copy\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-registration-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618394 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-var-lib-kubelet\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovn-node-metrics-cert\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618427 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-cnibin\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618461 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-kubelet\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-systemd\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-run\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618642 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-run\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618678 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.622686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.618768 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-kubernetes\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.620071 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.620138 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-sys\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.620187 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-run-netns\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.620232 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-systemd\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.620637 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-env-overrides\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.620748 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysconfig\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.621106 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-sys-fs\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.621237 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysctl-d\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.621284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-node-log\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.621331 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.621390 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovnkube-config\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.621459 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-systemd-units\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.621503 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-cni-bin\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.621810 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.622254 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovnkube-script-lib\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.622324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-registration-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.622373 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-var-lib-kubelet\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.623550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.622778 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/333ce92a-033d-45bc-bbfc-78ea69ff251c-host\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.622841 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-systemd\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.622880 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-cni-netd\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623170 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-device-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623218 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-host\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-run-ovn\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-kubelet\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623397 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-lib-modules\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623432 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-etc-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623495 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623524 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f562836-0b51-4578-a0d0-df5bba3702c8-socket-dir\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623555 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-modprobe-d\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623565 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-var-lib-openvswitch\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623584 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-slash\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623614 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623825 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-log-socket\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-sysctl-conf\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.624446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.623975 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/333ce92a-033d-45bc-bbfc-78ea69ff251c-serviceca\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.627183 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.627160 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-tmp\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.627325 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.627302 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-ovn-node-metrics-cert\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.628921 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.628898 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 08:39:18.629022 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.628929 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 08:39:18.629022 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.628943 2577 projected.go:196] Error preparing data for projected volume kube-api-access-8t9k6 for pod openshift-network-diagnostics/network-check-target-9w6g4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:18.629022 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.629003 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6 podName:dbeeae9e-b538-462d-a818-c8914cfdc325 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:19.128982034 +0000 UTC m=+2.962345574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8t9k6" (UniqueName: "kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6") pod "network-check-target-9w6g4" (UID: "dbeeae9e-b538-462d-a818-c8914cfdc325") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:18.633731 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.633546 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xmds\" (UniqueName: \"kubernetes.io/projected/affb99ae-08fe-4af4-8b48-6492cc1fd4e1-kube-api-access-7xmds\") pod \"ovnkube-node-jw7m5\" (UID: \"affb99ae-08fe-4af4-8b48-6492cc1fd4e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.634596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.634569 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8942n\" (UniqueName: \"kubernetes.io/projected/7f562836-0b51-4578-a0d0-df5bba3702c8-kube-api-access-8942n\") pod \"aws-ebs-csi-driver-node-mwnx6\" (UID: \"7f562836-0b51-4578-a0d0-df5bba3702c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.634703 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.634634 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l657j\" (UniqueName: \"kubernetes.io/projected/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-kube-api-access-l657j\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.634869 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.634825 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4z7t\" (UniqueName: \"kubernetes.io/projected/333ce92a-033d-45bc-bbfc-78ea69ff251c-kube-api-access-r4z7t\") pod \"node-ca-jmn4t\" (UID: \"333ce92a-033d-45bc-bbfc-78ea69ff251c\") " pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.636363 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.636273 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" event={"ID":"50583baa71de9d4d5e83da0f276b22a0","Type":"ContainerStarted","Data":"c7667c1b5f87ddf90c5657c2aca75d14764cd1dd906569d0fb954f3f2b5b4a39"} Sep 09 08:39:18.637745 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.637726 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4cf79c6-a064-4f3c-a9a8-204c611fa14e-etc-tuned\") pod \"tuned-tvfvr\" (UID: \"c4cf79c6-a064-4f3c-a9a8-204c611fa14e\") " pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.638086 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.637952 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" event={"ID":"581b21db7cc54e6fc2cafbc222baf6c8","Type":"ContainerStarted","Data":"3f0257ab9f4d4580cc39b5dfd87157e76879c3d4c7554b40107e3aeaf43ce649"} Sep 09 08:39:18.719262 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719229 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckzj4\" (UniqueName: \"kubernetes.io/projected/6a228400-e2ea-43dc-85c0-9bf945bb0872-kube-api-access-ckzj4\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.719262 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-netns\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.719484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-hostroot\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.719484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719323 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e4e33c67-cd89-4b31-946f-0792b3e9a131-agent-certs\") pod \"konnectivity-agent-9qxkj\" (UID: \"e4e33c67-cd89-4b31-946f-0792b3e9a131\") " pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:18.719484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719350 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.719484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719363 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-netns\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.719484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cni-binary-copy\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.719484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719384 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-hostroot\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.719484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719432 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.719484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719467 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-cnibin\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.719624 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719661 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-kubelet\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719722 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-cnibin\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719624 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-kubelet\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.719787 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret podName:6e9b855f-9ef8-4c2f-903d-492a4119d176 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:19.219759666 +0000 UTC m=+3.053123226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret") pod "global-pull-secret-syncer-c25ds" (UID: "6e9b855f-9ef8-4c2f-903d-492a4119d176") : object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719825 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e4e33c67-cd89-4b31-946f-0792b3e9a131-konnectivity-ca\") pod \"konnectivity-agent-9qxkj\" (UID: \"e4e33c67-cd89-4b31-946f-0792b3e9a131\") " pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719854 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-hosts-file\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-tmp-dir\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.719921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719922 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-system-cni-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719948 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-hosts-file\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719950 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-conf-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719996 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-conf-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720005 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cni-binary-copy\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720005 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.719999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-cni-multus\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720075 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-etc-kubernetes\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-system-cni-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a228400-e2ea-43dc-85c0-9bf945bb0872-host-slash\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720149 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-etc-kubernetes\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720152 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a228400-e2ea-43dc-85c0-9bf945bb0872-host-slash\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720162 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-cni-multus\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720208 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65a71213-3745-4fe0-8fd1-372771761d7a-cni-binary-copy\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-cni-bin\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720266 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65a71213-3745-4fe0-8fd1-372771761d7a-multus-daemon-config\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-k8s-cni-cncf-io\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720338 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-tmp-dir\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.720596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-cni-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720390 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc4jf\" (UniqueName: \"kubernetes.io/projected/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-kube-api-access-dc4jf\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720416 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn7kq\" (UniqueName: \"kubernetes.io/projected/65a71213-3745-4fe0-8fd1-372771761d7a-kube-api-access-tn7kq\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720428 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e4e33c67-cd89-4b31-946f-0792b3e9a131-konnectivity-ca\") pod \"konnectivity-agent-9qxkj\" (UID: \"e4e33c67-cd89-4b31-946f-0792b3e9a131\") " pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720451 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cbbp\" (UniqueName: \"kubernetes.io/projected/a4992530-e8b6-4ebd-bc09-04038870dfbf-kube-api-access-5cbbp\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cnibin\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6a228400-e2ea-43dc-85c0-9bf945bb0872-iptables-alerter-script\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720533 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-system-cni-dir\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720593 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-cni-dir\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-multus-certs\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e9b855f-9ef8-4c2f-903d-492a4119d176-kubelet-config\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720645 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-k8s-cni-cncf-io\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720681 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-socket-dir-parent\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e9b855f-9ef8-4c2f-903d-492a4119d176-dbus\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4h4w\" (UniqueName: \"kubernetes.io/projected/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-kube-api-access-k4h4w\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.721564 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720741 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-system-cni-dir\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720818 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cnibin\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720865 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-run-multus-certs\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720880 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65a71213-3745-4fe0-8fd1-372771761d7a-cni-binary-copy\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720882 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65a71213-3745-4fe0-8fd1-372771761d7a-multus-daemon-config\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720920 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e9b855f-9ef8-4c2f-903d-492a4119d176-kubelet-config\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.720934 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-host-var-lib-cni-bin\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.720998 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.721045 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e9b855f-9ef8-4c2f-903d-492a4119d176-dbus\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:18.721120 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs podName:a4992530-e8b6-4ebd-bc09-04038870dfbf nodeName:}" failed. No retries permitted until 2026-09-09 08:39:19.221094945 +0000 UTC m=+3.054458500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs") pod "network-metrics-daemon-wqxmh" (UID: "a4992530-e8b6-4ebd-bc09-04038870dfbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.721135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65a71213-3745-4fe0-8fd1-372771761d7a-multus-socket-dir-parent\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.721413 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6a228400-e2ea-43dc-85c0-9bf945bb0872-iptables-alerter-script\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.722452 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.721447 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.723114 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.722658 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e4e33c67-cd89-4b31-946f-0792b3e9a131-agent-certs\") pod \"konnectivity-agent-9qxkj\" (UID: \"e4e33c67-cd89-4b31-946f-0792b3e9a131\") " pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:18.729675 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.729626 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn7kq\" (UniqueName: \"kubernetes.io/projected/65a71213-3745-4fe0-8fd1-372771761d7a-kube-api-access-tn7kq\") pod \"multus-cqjwk\" (UID: \"65a71213-3745-4fe0-8fd1-372771761d7a\") " pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.730109 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.730084 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4h4w\" (UniqueName: \"kubernetes.io/projected/4b081bd8-2c98-4c0f-894b-3f8af9f71dbd-kube-api-access-k4h4w\") pod \"multus-additional-cni-plugins-q76kl\" (UID: \"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd\") " pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:18.730618 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.730596 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckzj4\" (UniqueName: \"kubernetes.io/projected/6a228400-e2ea-43dc-85c0-9bf945bb0872-kube-api-access-ckzj4\") pod \"iptables-alerter-tw4wg\" (UID: \"6a228400-e2ea-43dc-85c0-9bf945bb0872\") " pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.730982 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.730959 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc4jf\" (UniqueName: \"kubernetes.io/projected/7420c89d-6441-4350-9c6a-6a5b3fd8c2ca-kube-api-access-dc4jf\") pod \"node-resolver-lcbk4\" (UID: \"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca\") " pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.731078 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.730976 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cbbp\" (UniqueName: \"kubernetes.io/projected/a4992530-e8b6-4ebd-bc09-04038870dfbf-kube-api-access-5cbbp\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:18.817683 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.817604 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jmn4t" Sep 09 08:39:18.829447 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.829426 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" Sep 09 08:39:18.831040 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:18.830892 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod333ce92a_033d_45bc_bbfc_78ea69ff251c.slice/crio-0dddb5267157dbb2dc04defcdc56e0314801b36e25d65670bf21679d5da8c5fe WatchSource:0}: Error finding container 0dddb5267157dbb2dc04defcdc56e0314801b36e25d65670bf21679d5da8c5fe: Status 404 returned error can't find the container with id 0dddb5267157dbb2dc04defcdc56e0314801b36e25d65670bf21679d5da8c5fe Sep 09 08:39:18.840447 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.840422 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" Sep 09 08:39:18.842315 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:18.842288 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f562836_0b51_4578_a0d0_df5bba3702c8.slice/crio-1c2afc159826ebed1f5783ce28ea67b613b728ff9f1d113d42525c0f0acc9e8c WatchSource:0}: Error finding container 1c2afc159826ebed1f5783ce28ea67b613b728ff9f1d113d42525c0f0acc9e8c: Status 404 returned error can't find the container with id 1c2afc159826ebed1f5783ce28ea67b613b728ff9f1d113d42525c0f0acc9e8c Sep 09 08:39:18.848195 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.848171 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:18.855634 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.855609 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cqjwk" Sep 09 08:39:18.865945 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.865895 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:18.875778 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.875755 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tw4wg" Sep 09 08:39:18.883135 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.883117 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lcbk4" Sep 09 08:39:18.890266 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:18.890249 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q76kl" Sep 09 08:39:19.224259 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:19.224184 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:19.224259 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:19.224228 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:19.224259 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:19.224256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:19.224468 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.224342 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:19.224468 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.224366 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 08:39:19.224468 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.224386 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 08:39:19.224468 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.224390 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret podName:6e9b855f-9ef8-4c2f-903d-492a4119d176 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:20.224378386 +0000 UTC m=+4.057741925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret") pod "global-pull-secret-syncer-c25ds" (UID: "6e9b855f-9ef8-4c2f-903d-492a4119d176") : object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:19.224468 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.224396 2577 projected.go:196] Error preparing data for projected volume kube-api-access-8t9k6 for pod openshift-network-diagnostics/network-check-target-9w6g4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:19.224468 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.224344 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:19.224468 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.224444 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6 podName:dbeeae9e-b538-462d-a818-c8914cfdc325 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:20.224426339 +0000 UTC m=+4.057789892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t9k6" (UniqueName: "kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6") pod "network-check-target-9w6g4" (UID: "dbeeae9e-b538-462d-a818-c8914cfdc325") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:19.224468 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.224462 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs podName:a4992530-e8b6-4ebd-bc09-04038870dfbf nodeName:}" failed. No retries permitted until 2026-09-09 08:39:20.224455326 +0000 UTC m=+4.057818865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs") pod "network-metrics-daemon-wqxmh" (UID: "a4992530-e8b6-4ebd-bc09-04038870dfbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:19.563057 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:19.562951 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 08:34:19 +0000 UTC" deadline="2028-05-23 15:58:22.533724748 +0000 UTC" Sep 09 08:39:19.563057 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:19.563010 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14935h19m2.970717855s" Sep 09 08:39:19.627988 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:19.627947 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:19.628148 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:19.628081 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:19.641117 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:19.641079 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" event={"ID":"7f562836-0b51-4578-a0d0-df5bba3702c8","Type":"ContainerStarted","Data":"1c2afc159826ebed1f5783ce28ea67b613b728ff9f1d113d42525c0f0acc9e8c"} Sep 09 08:39:19.642490 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:19.642464 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jmn4t" event={"ID":"333ce92a-033d-45bc-bbfc-78ea69ff251c","Type":"ContainerStarted","Data":"0dddb5267157dbb2dc04defcdc56e0314801b36e25d65670bf21679d5da8c5fe"} Sep 09 08:39:19.929349 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:19.929318 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e33c67_cd89_4b31_946f_0792b3e9a131.slice/crio-9926ebe45f902ac6b14d448d5f4ab23b52bb991a226acdc887a3acb15d06548b WatchSource:0}: Error finding container 9926ebe45f902ac6b14d448d5f4ab23b52bb991a226acdc887a3acb15d06548b: Status 404 returned error can't find the container with id 9926ebe45f902ac6b14d448d5f4ab23b52bb991a226acdc887a3acb15d06548b Sep 09 08:39:19.934101 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:19.934061 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaffb99ae_08fe_4af4_8b48_6492cc1fd4e1.slice/crio-6f0b4c8ac5a8ca6e661e5bb9c8959afa788d83c447025de55c5fc1c4821001a7 WatchSource:0}: Error finding container 6f0b4c8ac5a8ca6e661e5bb9c8959afa788d83c447025de55c5fc1c4821001a7: Status 404 returned error can't find the container with id 6f0b4c8ac5a8ca6e661e5bb9c8959afa788d83c447025de55c5fc1c4821001a7 Sep 09 08:39:19.941012 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:19.940586 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cf79c6_a064_4f3c_a9a8_204c611fa14e.slice/crio-886d73274f8666e432a71e57531cae04edf556dc6c90fd9d14596476da5e2a6c WatchSource:0}: Error finding container 886d73274f8666e432a71e57531cae04edf556dc6c90fd9d14596476da5e2a6c: Status 404 returned error can't find the container with id 886d73274f8666e432a71e57531cae04edf556dc6c90fd9d14596476da5e2a6c Sep 09 08:39:19.941012 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:19.940974 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a71213_3745_4fe0_8fd1_372771761d7a.slice/crio-a5a93d838e2262637bfb8a23195322f728b173b84137cd2aa632b8f047eda1f6 WatchSource:0}: Error finding container a5a93d838e2262637bfb8a23195322f728b173b84137cd2aa632b8f047eda1f6: Status 404 returned error can't find the container with id a5a93d838e2262637bfb8a23195322f728b173b84137cd2aa632b8f047eda1f6 Sep 09 08:39:19.941945 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:19.941830 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b081bd8_2c98_4c0f_894b_3f8af9f71dbd.slice/crio-65929fc68b5c5f1fb0b1e89dbf81faa3d9ba9a96b2a36207c94f2fa60a34e7c1 WatchSource:0}: Error finding container 65929fc68b5c5f1fb0b1e89dbf81faa3d9ba9a96b2a36207c94f2fa60a34e7c1: Status 404 returned error can't find the container with id 65929fc68b5c5f1fb0b1e89dbf81faa3d9ba9a96b2a36207c94f2fa60a34e7c1 Sep 09 08:39:19.945111 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:39:19.945000 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7420c89d_6441_4350_9c6a_6a5b3fd8c2ca.slice/crio-efd1f90f34a0251ea2e48796da4c0ba80145b523f3de1edc486d9ac05fad062a WatchSource:0}: Error finding container efd1f90f34a0251ea2e48796da4c0ba80145b523f3de1edc486d9ac05fad062a: Status 404 returned error can't find the container with id efd1f90f34a0251ea2e48796da4c0ba80145b523f3de1edc486d9ac05fad062a Sep 09 08:39:20.232424 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.232246 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:20.232424 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.232315 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:20.232424 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.232358 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:20.232684 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.232439 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 08:39:20.232684 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.232468 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:20.232684 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.232473 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:20.232684 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.232465 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 08:39:20.232684 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.232531 2577 projected.go:196] Error preparing data for projected volume kube-api-access-8t9k6 for pod openshift-network-diagnostics/network-check-target-9w6g4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:20.232684 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.232534 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs podName:a4992530-e8b6-4ebd-bc09-04038870dfbf nodeName:}" failed. No retries permitted until 2026-09-09 08:39:22.232513096 +0000 UTC m=+6.065876662 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs") pod "network-metrics-daemon-wqxmh" (UID: "a4992530-e8b6-4ebd-bc09-04038870dfbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:20.232684 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.232554 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret podName:6e9b855f-9ef8-4c2f-903d-492a4119d176 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:22.232544081 +0000 UTC m=+6.065907626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret") pod "global-pull-secret-syncer-c25ds" (UID: "6e9b855f-9ef8-4c2f-903d-492a4119d176") : object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:20.232684 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.232570 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6 podName:dbeeae9e-b538-462d-a818-c8914cfdc325 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:22.232560863 +0000 UTC m=+6.065924450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t9k6" (UniqueName: "kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6") pod "network-check-target-9w6g4" (UID: "dbeeae9e-b538-462d-a818-c8914cfdc325") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:20.564153 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.563843 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 08:34:19 +0000 UTC" deadline="2028-03-07 06:14:40.934896771 +0000 UTC" Sep 09 08:39:20.564153 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.563887 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13077h35m20.371015763s" Sep 09 08:39:20.631743 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.631203 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:20.631743 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.631203 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:20.631743 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.631336 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:20.631743 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:20.631375 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:20.661874 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.661563 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" event={"ID":"581b21db7cc54e6fc2cafbc222baf6c8","Type":"ContainerStarted","Data":"e65718f4928c067f6333f8099066cdcaaf6633471f13ec3f6ff081a213afd868"} Sep 09 08:39:20.664429 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.664365 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lcbk4" event={"ID":"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca","Type":"ContainerStarted","Data":"efd1f90f34a0251ea2e48796da4c0ba80145b523f3de1edc486d9ac05fad062a"} Sep 09 08:39:20.666672 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.666643 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q76kl" event={"ID":"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd","Type":"ContainerStarted","Data":"65929fc68b5c5f1fb0b1e89dbf81faa3d9ba9a96b2a36207c94f2fa60a34e7c1"} Sep 09 08:39:20.668461 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.668435 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" event={"ID":"c4cf79c6-a064-4f3c-a9a8-204c611fa14e","Type":"ContainerStarted","Data":"886d73274f8666e432a71e57531cae04edf556dc6c90fd9d14596476da5e2a6c"} Sep 09 08:39:20.672953 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.672919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"6f0b4c8ac5a8ca6e661e5bb9c8959afa788d83c447025de55c5fc1c4821001a7"} Sep 09 08:39:20.675270 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.675247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tw4wg" event={"ID":"6a228400-e2ea-43dc-85c0-9bf945bb0872","Type":"ContainerStarted","Data":"e00947ee96fdc0b6e25cc6db80f03d3db1d0adc7bee68ec6b4f60c3fed0b1906"} Sep 09 08:39:20.679255 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.679210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cqjwk" event={"ID":"65a71213-3745-4fe0-8fd1-372771761d7a","Type":"ContainerStarted","Data":"a5a93d838e2262637bfb8a23195322f728b173b84137cd2aa632b8f047eda1f6"} Sep 09 08:39:20.694798 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.694715 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9qxkj" event={"ID":"e4e33c67-cd89-4b31-946f-0792b3e9a131","Type":"ContainerStarted","Data":"9926ebe45f902ac6b14d448d5f4ab23b52bb991a226acdc887a3acb15d06548b"} Sep 09 08:39:20.699978 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.699209 2577 generic.go:356] "Generic (PLEG): container finished" podID="50583baa71de9d4d5e83da0f276b22a0" containerID="7a7accb08bc82d1b66846d01f70c1922e90d8e570f3debbc68e6e459ef306687" exitCode=0 Sep 09 08:39:20.699978 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.699247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" event={"ID":"50583baa71de9d4d5e83da0f276b22a0","Type":"ContainerDied","Data":"7a7accb08bc82d1b66846d01f70c1922e90d8e570f3debbc68e6e459ef306687"} Sep 09 08:39:20.716132 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:20.715318 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-59.ec2.internal" podStartSLOduration=2.71529872 podStartE2EDuration="2.71529872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:20.679068603 +0000 UTC m=+4.512432174" watchObservedRunningTime="2026-09-09 08:39:20.71529872 +0000 UTC m=+4.548662284" Sep 09 08:39:21.628118 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:21.628078 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:21.628619 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:21.628227 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:21.737035 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:21.736891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" event={"ID":"50583baa71de9d4d5e83da0f276b22a0","Type":"ContainerStarted","Data":"47bb2b60ecc0f5474f7a662cff5f184abe23b2a0dc43fe7d16d8b5fb526298c4"} Sep 09 08:39:21.754790 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:21.754129 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-59.ec2.internal" podStartSLOduration=3.754109845 podStartE2EDuration="3.754109845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:21.753600725 +0000 UTC m=+5.586964289" watchObservedRunningTime="2026-09-09 08:39:21.754109845 +0000 UTC m=+5.587473410" Sep 09 08:39:22.253235 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:22.253126 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:22.253235 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:22.253197 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:22.253465 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:22.253245 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:22.253465 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.253397 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 08:39:22.253465 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.253419 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 08:39:22.253465 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.253433 2577 projected.go:196] Error preparing data for projected volume kube-api-access-8t9k6 for pod openshift-network-diagnostics/network-check-target-9w6g4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:22.253662 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.253495 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6 podName:dbeeae9e-b538-462d-a818-c8914cfdc325 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:26.253475707 +0000 UTC m=+10.086839268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t9k6" (UniqueName: "kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6") pod "network-check-target-9w6g4" (UID: "dbeeae9e-b538-462d-a818-c8914cfdc325") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:22.253999 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.253963 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:22.254142 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.254028 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs podName:a4992530-e8b6-4ebd-bc09-04038870dfbf nodeName:}" failed. No retries permitted until 2026-09-09 08:39:26.254010211 +0000 UTC m=+10.087373755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs") pod "network-metrics-daemon-wqxmh" (UID: "a4992530-e8b6-4ebd-bc09-04038870dfbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:22.254233 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.254121 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:22.254233 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.254191 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret podName:6e9b855f-9ef8-4c2f-903d-492a4119d176 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:26.254178595 +0000 UTC m=+10.087542141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret") pod "global-pull-secret-syncer-c25ds" (UID: "6e9b855f-9ef8-4c2f-903d-492a4119d176") : object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:22.628784 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:22.628695 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:22.630291 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.630252 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:22.630946 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:22.630680 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:22.630946 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:22.630798 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:23.628475 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:23.628380 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:23.628680 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:23.628527 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:24.629245 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:24.628689 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:24.629245 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:24.628693 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:24.629245 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:24.628856 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:24.629245 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:24.628910 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:25.628728 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:25.628643 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:25.628878 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:25.628750 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:26.289054 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:26.289006 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:26.289079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:26.289119 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.289232 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.289236 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.289252 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.289264 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.289279 2577 projected.go:196] Error preparing data for projected volume kube-api-access-8t9k6 for pod openshift-network-diagnostics/network-check-target-9w6g4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.289298 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret podName:6e9b855f-9ef8-4c2f-903d-492a4119d176 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:34.28928093 +0000 UTC m=+18.122644474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret") pod "global-pull-secret-syncer-c25ds" (UID: "6e9b855f-9ef8-4c2f-903d-492a4119d176") : object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.289327 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs podName:a4992530-e8b6-4ebd-bc09-04038870dfbf nodeName:}" failed. No retries permitted until 2026-09-09 08:39:34.289308421 +0000 UTC m=+18.122671969 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs") pod "network-metrics-daemon-wqxmh" (UID: "a4992530-e8b6-4ebd-bc09-04038870dfbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:26.289504 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.289347 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6 podName:dbeeae9e-b538-462d-a818-c8914cfdc325 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:34.289337198 +0000 UTC m=+18.122700736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t9k6" (UniqueName: "kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6") pod "network-check-target-9w6g4" (UID: "dbeeae9e-b538-462d-a818-c8914cfdc325") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:26.629893 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:26.629569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:26.629893 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.629690 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:26.629893 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:26.629752 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:26.629893 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:26.629818 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:27.628179 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:27.628151 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:27.628564 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:27.628259 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:28.628048 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:28.628012 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:28.628232 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:28.628012 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:28.628232 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:28.628166 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:28.628640 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:28.628260 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:29.628066 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:29.628033 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:29.628266 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:29.628171 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:30.629058 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:30.628968 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:30.629058 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:30.629003 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:30.629479 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:30.629096 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:30.629479 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:30.629279 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:31.628357 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:31.628325 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:31.628547 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:31.628452 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:32.628671 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:32.628631 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:32.629139 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:32.628631 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:32.629139 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:32.628804 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:32.629139 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:32.628869 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:33.628939 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:33.628905 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:33.629391 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:33.629043 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:34.348908 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:34.348874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:34.349113 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:34.348934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:34.349113 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:34.348991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:34.349113 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.349096 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 08:39:34.349262 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.349112 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:34.349262 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.349132 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 08:39:34.349262 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.349146 2577 projected.go:196] Error preparing data for projected volume kube-api-access-8t9k6 for pod openshift-network-diagnostics/network-check-target-9w6g4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:34.349262 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.349099 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:34.349262 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.349183 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs podName:a4992530-e8b6-4ebd-bc09-04038870dfbf nodeName:}" failed. No retries permitted until 2026-09-09 08:39:50.349160992 +0000 UTC m=+34.182524533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs") pod "network-metrics-daemon-wqxmh" (UID: "a4992530-e8b6-4ebd-bc09-04038870dfbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:34.349262 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.349229 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6 podName:dbeeae9e-b538-462d-a818-c8914cfdc325 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:50.349213006 +0000 UTC m=+34.182576547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t9k6" (UniqueName: "kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6") pod "network-check-target-9w6g4" (UID: "dbeeae9e-b538-462d-a818-c8914cfdc325") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:34.349262 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.349245 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret podName:6e9b855f-9ef8-4c2f-903d-492a4119d176 nodeName:}" failed. No retries permitted until 2026-09-09 08:39:50.349236676 +0000 UTC m=+34.182600214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret") pod "global-pull-secret-syncer-c25ds" (UID: "6e9b855f-9ef8-4c2f-903d-492a4119d176") : object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:34.628769 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:34.628687 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:34.628934 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:34.628687 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:34.628934 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.628830 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:34.628934 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:34.628896 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:35.628359 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:35.628316 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:35.628550 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:35.628457 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:36.628636 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:36.628601 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:36.629108 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:36.628691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:36.629108 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:36.628714 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:36.629108 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:36.628812 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:37.629198 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.628054 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:37.629198 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:37.628408 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:37.769068 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.769035 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" event={"ID":"c4cf79c6-a064-4f3c-a9a8-204c611fa14e","Type":"ContainerStarted","Data":"d61adb9e99a6bd018dac02046ca31dacbdf136146f37a1c96af61bca658d234f"} Sep 09 08:39:37.771518 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.771494 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"7f6a91a2d21b8dc5f97373a267c95f52130f54669012c4f25b8ecf0242861028"} Sep 09 08:39:37.771620 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.771572 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"b20d5d5fd02a57c2d6e4018031977e9652ebdc94aa6943d360dd3c50e90bd06e"} Sep 09 08:39:37.773276 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.773242 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cqjwk" event={"ID":"65a71213-3745-4fe0-8fd1-372771761d7a","Type":"ContainerStarted","Data":"00099de5754562c55901607c59b81b565e64cab10927a50c8f2ffc3aaef652de"} Sep 09 08:39:37.774733 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.774703 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9qxkj" event={"ID":"e4e33c67-cd89-4b31-946f-0792b3e9a131","Type":"ContainerStarted","Data":"fc91058f40de82c3de88aba5e2dc5a988b248fcf6081101256b8579779a7a582"} Sep 09 08:39:37.776456 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.776428 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" event={"ID":"7f562836-0b51-4578-a0d0-df5bba3702c8","Type":"ContainerStarted","Data":"d84bba9a3aa3c2da347df54a611724819eef1a0938627e1cf8fea408d71c203d"} Sep 09 08:39:37.778049 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.777927 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jmn4t" event={"ID":"333ce92a-033d-45bc-bbfc-78ea69ff251c","Type":"ContainerStarted","Data":"b2d7308b835aaf7d259b7de24d884e994b06258148b5df4b5da931bacbbad558"} Sep 09 08:39:37.779601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.779570 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lcbk4" event={"ID":"7420c89d-6441-4350-9c6a-6a5b3fd8c2ca","Type":"ContainerStarted","Data":"c79b7586eda767a7e3b7fe6e9bc7e1bb5d7433be0fb80e808719e5310b9f0240"} Sep 09 08:39:37.781289 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.781265 2577 generic.go:356] "Generic (PLEG): container finished" podID="4b081bd8-2c98-4c0f-894b-3f8af9f71dbd" containerID="1c26355243275eedebe75dd6e2c4ee65fc8108ce7c1e926a9b588666be8555ed" exitCode=0 Sep 09 08:39:37.781378 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.781302 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q76kl" event={"ID":"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd","Type":"ContainerDied","Data":"1c26355243275eedebe75dd6e2c4ee65fc8108ce7c1e926a9b588666be8555ed"} Sep 09 08:39:37.784250 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.784133 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tvfvr" podStartSLOduration=3.275373492 podStartE2EDuration="20.784115543s" totalImagesPullingTime="17.508742051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:37.783844535 +0000 UTC m=+21.617208108" watchObservedRunningTime="2026-09-09 08:39:37.784115543 +0000 UTC m=+21.617479106" Sep 09 08:39:37.797939 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.797883 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lcbk4" podStartSLOduration=3.289012336 podStartE2EDuration="20.797868068s" totalImagesPullingTime="17.508855732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:37.797355967 +0000 UTC m=+21.630719548" watchObservedRunningTime="2026-09-09 08:39:37.797868068 +0000 UTC m=+21.631231669" Sep 09 08:39:37.831374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.831330 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jmn4t" podStartSLOduration=5.511782309 podStartE2EDuration="20.831313615s" totalImagesPullingTime="15.319531306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:37.830786034 +0000 UTC m=+21.664149593" watchObservedRunningTime="2026-09-09 08:39:37.831313615 +0000 UTC m=+21.664677174" Sep 09 08:39:37.847559 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.847078 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cqjwk" podStartSLOduration=3.274448572 podStartE2EDuration="20.847056916s" totalImagesPullingTime="17.572608344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:37.846438956 +0000 UTC m=+21.679802517" watchObservedRunningTime="2026-09-09 08:39:37.847056916 +0000 UTC m=+21.680420475" Sep 09 08:39:37.863048 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.862999 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9qxkj" podStartSLOduration=10.324630455 podStartE2EDuration="20.862968419s" totalImagesPullingTime="10.538337964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:37.862610448 +0000 UTC m=+21.695974004" watchObservedRunningTime="2026-09-09 08:39:37.862968419 +0000 UTC m=+21.696331978" Sep 09 08:39:37.993282 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.993196 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:37.993682 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:37.993666 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:38.628948 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.628918 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:38.629451 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:38.629071 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:38.629451 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.629131 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:38.629451 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:38.629235 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:38.787677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.787587 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" event={"ID":"7f562836-0b51-4578-a0d0-df5bba3702c8","Type":"ContainerStarted","Data":"51b786a73c09fb87622aa3328ffbb4fcba2a0d82f3b5b07c0837954d6f7f886e"} Sep 09 08:39:38.800053 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.800002 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 08:39:38.800053 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.800022 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"82b970c8954814d1f96dbc077fcb40cbcdd9ac55549e7a9e40771ea0aa734954"} Sep 09 08:39:38.800278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.800102 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"bd642847e0640a601e094aef27d25433a9bb1aed9e56ae3b219ad7aece32cf6f"} Sep 09 08:39:38.800278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.800148 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"ea50cab03a97ddf33524e2b062ff3e7ce30f2083d0ef5d27e010b99cb0162e23"} Sep 09 08:39:38.800278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.800190 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"db9621d7d0a0f34d70712a1579b6ba740f43d5a407eba294c47ba3c60f24cb56"} Sep 09 08:39:38.801808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.801780 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tw4wg" event={"ID":"6a228400-e2ea-43dc-85c0-9bf945bb0872","Type":"ContainerStarted","Data":"05ca303f0d43714a30c40ea78d5839a012ba1f2cff9c3c59aa4db3f25a971439"} Sep 09 08:39:38.802668 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.802625 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:38.803129 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.803110 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9qxkj" Sep 09 08:39:38.816355 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:38.816311 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tw4wg" podStartSLOduration=4.3146495 podStartE2EDuration="21.81629877s" totalImagesPullingTime="17.50164927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:38.815957932 +0000 UTC m=+22.649321490" watchObservedRunningTime="2026-09-09 08:39:38.81629877 +0000 UTC m=+22.649662329" Sep 09 08:39:39.568527 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:39.568400 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T08:39:38.80002236Z","UUID":"75c33931-bf33-45e3-88c5-f3f97824a859","Handler":null,"Name":"","Endpoint":""} Sep 09 08:39:39.570129 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:39.570091 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 08:39:39.570129 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:39.570130 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 08:39:39.628099 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:39.628067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:39.628273 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:39.628183 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:40.629017 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:40.628988 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:40.629390 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:40.629089 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:40.629390 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:40.629149 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:40.629390 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:40.629265 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:40.809654 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:40.809617 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" event={"ID":"7f562836-0b51-4578-a0d0-df5bba3702c8","Type":"ContainerStarted","Data":"17bc2d62b7e96440a7ecfdaaf8faee3fb8377276ca472b97ceefa08ff57f4bd7"} Sep 09 08:39:40.813424 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:40.813383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"79293122990feee266239bf6f03285e2c7e0ab36c206a56046fe682e15623cda"} Sep 09 08:39:40.828552 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:40.828506 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mwnx6" podStartSLOduration=3.290688676 podStartE2EDuration="23.828494547s" totalImagesPullingTime="20.537805871s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:40.828357315 +0000 UTC m=+24.661720898" watchObservedRunningTime="2026-09-09 08:39:40.828494547 +0000 UTC m=+24.661858106" Sep 09 08:39:41.628770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:41.628740 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:41.628932 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:41.628848 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:42.628388 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:42.628205 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:42.628955 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:42.628255 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:42.628955 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:42.628478 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:42.628955 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:42.628521 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:42.819171 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:42.819135 2577 generic.go:356] "Generic (PLEG): container finished" podID="4b081bd8-2c98-4c0f-894b-3f8af9f71dbd" containerID="fe83ff83b49823517a0e5987d155e8cb22030fca72376866d3e3e69c36c0e6ab" exitCode=0 Sep 09 08:39:42.819338 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:42.819222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q76kl" event={"ID":"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd","Type":"ContainerDied","Data":"fe83ff83b49823517a0e5987d155e8cb22030fca72376866d3e3e69c36c0e6ab"} Sep 09 08:39:42.822958 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:42.822939 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" event={"ID":"affb99ae-08fe-4af4-8b48-6492cc1fd4e1","Type":"ContainerStarted","Data":"c5978121266f7fd4a014c9c1795a1688ccf71c570912e76afc8a946a909ae40f"} Sep 09 08:39:42.823223 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:42.823205 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:42.838088 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:42.838071 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:42.863287 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:42.863251 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" podStartSLOduration=8.288812954 podStartE2EDuration="25.863238145s" totalImagesPullingTime="17.574425191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:42.861423925 +0000 UTC m=+26.694787483" watchObservedRunningTime="2026-09-09 08:39:42.863238145 +0000 UTC m=+26.696601703" Sep 09 08:39:43.628681 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:43.628645 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:43.629295 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:43.628754 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:43.825637 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:43.825607 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:43.825848 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:43.825823 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:43.843886 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:43.843857 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:39:44.172751 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:44.172506 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c25ds"] Sep 09 08:39:44.172887 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:44.172776 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:44.172928 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:44.172888 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:44.175554 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:44.175527 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wqxmh"] Sep 09 08:39:44.175691 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:44.175639 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:44.175771 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:44.175749 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:44.176170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:44.176149 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9w6g4"] Sep 09 08:39:44.176260 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:44.176243 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:44.176362 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:44.176337 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:44.829406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:44.829375 2577 generic.go:356] "Generic (PLEG): container finished" podID="4b081bd8-2c98-4c0f-894b-3f8af9f71dbd" containerID="6b780aa09ef0f8f69e87006db3502c7b8d0801885aacf7e6374efab9a3c1d137" exitCode=0 Sep 09 08:39:44.829883 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:44.829461 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q76kl" event={"ID":"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd","Type":"ContainerDied","Data":"6b780aa09ef0f8f69e87006db3502c7b8d0801885aacf7e6374efab9a3c1d137"} Sep 09 08:39:45.628781 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:45.628744 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:45.628993 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:45.628844 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:45.628993 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:45.628745 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:45.628993 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:45.628745 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:45.628993 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:45.628918 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:45.628993 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:45.628961 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:45.834121 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:45.834031 2577 generic.go:356] "Generic (PLEG): container finished" podID="4b081bd8-2c98-4c0f-894b-3f8af9f71dbd" containerID="6af7efa5f4068f16b7ae17f8616988f68b2523a63124885d53dc190a04fd3be3" exitCode=0 Sep 09 08:39:45.834561 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:45.834122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q76kl" event={"ID":"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd","Type":"ContainerDied","Data":"6af7efa5f4068f16b7ae17f8616988f68b2523a63124885d53dc190a04fd3be3"} Sep 09 08:39:47.628243 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:47.628016 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:47.628664 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:47.628043 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:47.628664 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:47.628328 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:47.628664 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:47.628049 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:47.628664 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:47.628417 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:47.628664 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:47.628525 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:49.627939 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:49.627906 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:49.628529 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:49.628024 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:49.628529 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:49.628041 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:49.628529 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:49.628035 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:49.628529 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:49.628095 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:49.628529 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:49.628158 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:50.361678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:50.361649 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:50.361803 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:50.361710 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:50.361803 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:50.361746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:50.361910 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:50.361877 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:50.361910 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:50.361886 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:50.362032 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:50.361905 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 08:39:50.362032 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:50.361930 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 08:39:50.362032 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:50.361943 2577 projected.go:196] Error preparing data for projected volume kube-api-access-8t9k6 for pod openshift-network-diagnostics/network-check-target-9w6g4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:50.362032 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:50.361943 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret podName:6e9b855f-9ef8-4c2f-903d-492a4119d176 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:22.361924816 +0000 UTC m=+66.195288355 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret") pod "global-pull-secret-syncer-c25ds" (UID: "6e9b855f-9ef8-4c2f-903d-492a4119d176") : object "kube-system"/"original-pull-secret" not registered Sep 09 08:39:50.362032 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:50.362001 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs podName:a4992530-e8b6-4ebd-bc09-04038870dfbf nodeName:}" failed. No retries permitted until 2026-09-09 08:40:22.361990787 +0000 UTC m=+66.195354330 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs") pod "network-metrics-daemon-wqxmh" (UID: "a4992530-e8b6-4ebd-bc09-04038870dfbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 08:39:50.362032 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:50.362016 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6 podName:dbeeae9e-b538-462d-a818-c8914cfdc325 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:22.362007766 +0000 UTC m=+66.195371303 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t9k6" (UniqueName: "kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6") pod "network-check-target-9w6g4" (UID: "dbeeae9e-b538-462d-a818-c8914cfdc325") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 08:39:50.846458 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:50.846428 2577 generic.go:356] "Generic (PLEG): container finished" podID="4b081bd8-2c98-4c0f-894b-3f8af9f71dbd" containerID="057b7a7186f6bcbcb7d29c2ef4d48c90a0a9bea68b7176c66d7b4e072c29774b" exitCode=0 Sep 09 08:39:50.846804 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:50.846477 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q76kl" event={"ID":"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd","Type":"ContainerDied","Data":"057b7a7186f6bcbcb7d29c2ef4d48c90a0a9bea68b7176c66d7b4e072c29774b"} Sep 09 08:39:51.628192 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:51.628160 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:51.628359 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:51.628196 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:51.628359 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:51.628254 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:51.628359 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:51.628273 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:51.628359 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:51.628315 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:51.628493 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:51.628390 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:51.853181 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:51.853147 2577 generic.go:356] "Generic (PLEG): container finished" podID="4b081bd8-2c98-4c0f-894b-3f8af9f71dbd" containerID="7d6d1b30f9aa82e651245ab927f6e549a6dfa83e1e8599dbb0d2d889c7513de1" exitCode=0 Sep 09 08:39:51.853745 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:51.853200 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q76kl" event={"ID":"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd","Type":"ContainerDied","Data":"7d6d1b30f9aa82e651245ab927f6e549a6dfa83e1e8599dbb0d2d889c7513de1"} Sep 09 08:39:52.858692 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:52.858661 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q76kl" event={"ID":"4b081bd8-2c98-4c0f-894b-3f8af9f71dbd","Type":"ContainerStarted","Data":"83d12e98b6f58e44d31d3ec8ddeb369f1d4816882192fc2bd080b15c43fcbe62"} Sep 09 08:39:52.879891 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:52.879851 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q76kl" podStartSLOduration=7.922322287 podStartE2EDuration="35.879836129s" totalImagesPullingTime="27.957513842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:39:52.878174713 +0000 UTC m=+36.711538271" watchObservedRunningTime="2026-09-09 08:39:52.879836129 +0000 UTC m=+36.713199687" Sep 09 08:39:53.628547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:53.628511 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:53.628729 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:53.628511 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:53.628729 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:53.628618 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:53.628729 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:53.628702 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:53.628729 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:53.628514 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:53.628937 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:53.628811 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:55.628155 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:55.628125 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:55.628529 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:55.628125 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:55.628529 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:55.628221 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:55.628529 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:55.628306 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:55.628529 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:55.628125 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:55.628529 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:55.628389 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:57.628947 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:57.628908 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:57.628947 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:57.628941 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:57.629427 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:57.628941 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:57.629427 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:57.629024 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:57.629427 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:57.629098 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:57.629427 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:57.629161 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:39:59.628967 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:59.628931 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:39:59.629370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:59.628930 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:39:59.629370 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:59.629047 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:39:59.629370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:39:59.628930 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:39:59.629370 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:59.629130 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:39:59.629370 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:39:59.629184 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:40:01.628608 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:01.628574 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:40:01.629012 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:01.628574 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:40:01.629012 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:01.628776 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:40:01.629012 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:01.628672 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:40:01.629012 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:01.628577 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:40:01.629012 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:01.628862 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:40:03.627896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:03.627863 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:40:03.627896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:03.627882 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:40:03.627896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:03.627905 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:40:03.628462 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:03.628010 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:40:03.628462 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:03.628127 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:40:03.628462 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:03.628196 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:40:05.628951 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:05.628913 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:40:05.629449 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:05.628913 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:40:05.629449 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:05.629035 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wqxmh" podUID="a4992530-e8b6-4ebd-bc09-04038870dfbf" Sep 09 08:40:05.629449 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:05.628913 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:40:05.629449 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:05.629116 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9w6g4" podUID="dbeeae9e-b538-462d-a818-c8914cfdc325" Sep 09 08:40:05.629449 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:05.629159 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c25ds" podUID="6e9b855f-9ef8-4c2f-903d-492a4119d176" Sep 09 08:40:07.488041 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.488009 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-59.ec2.internal" event="NodeReady" Sep 09 08:40:07.488402 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.488127 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 08:40:07.534553 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.534516 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c9gmd"] Sep 09 08:40:07.536617 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.536596 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.539207 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.539186 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 08:40:07.539325 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.539307 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 08:40:07.539429 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.539412 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zfltp\"" Sep 09 08:40:07.541509 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.541492 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s7mrr"] Sep 09 08:40:07.544396 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.544380 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s7mrr" Sep 09 08:40:07.546489 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.546470 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c9gmd"] Sep 09 08:40:07.546890 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.546869 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 08:40:07.547200 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.547178 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j8mw2\"" Sep 09 08:40:07.547296 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.547205 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 08:40:07.547296 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.547209 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 08:40:07.553257 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.553237 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s7mrr"] Sep 09 08:40:07.628264 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.628227 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:40:07.628427 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.628268 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:40:07.628427 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.628272 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:40:07.631635 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.631610 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 08:40:07.631755 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.631615 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 08:40:07.631828 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.631811 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 08:40:07.631949 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.631932 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4bf2n\"" Sep 09 08:40:07.632078 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.632056 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dtwcb\"" Sep 09 08:40:07.632517 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.632505 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 08:40:07.686909 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.686879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/294058ff-720d-4cb7-a1da-cece1197a475-cert\") pod \"ingress-canary-s7mrr\" (UID: \"294058ff-720d-4cb7-a1da-cece1197a475\") " pod="openshift-ingress-canary/ingress-canary-s7mrr" Sep 09 08:40:07.687049 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.686916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf2d8\" (UniqueName: \"kubernetes.io/projected/294058ff-720d-4cb7-a1da-cece1197a475-kube-api-access-bf2d8\") pod \"ingress-canary-s7mrr\" (UID: \"294058ff-720d-4cb7-a1da-cece1197a475\") " pod="openshift-ingress-canary/ingress-canary-s7mrr" Sep 09 08:40:07.687049 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.686952 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-metrics-tls\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.687049 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.687032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-tmp-dir\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.687168 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.687053 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw2pm\" (UniqueName: \"kubernetes.io/projected/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-kube-api-access-hw2pm\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.687168 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.687073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-config-volume\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.787633 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.787601 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-config-volume\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.787810 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.787648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/294058ff-720d-4cb7-a1da-cece1197a475-cert\") pod \"ingress-canary-s7mrr\" (UID: \"294058ff-720d-4cb7-a1da-cece1197a475\") " pod="openshift-ingress-canary/ingress-canary-s7mrr" Sep 09 08:40:07.787810 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.787668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf2d8\" (UniqueName: \"kubernetes.io/projected/294058ff-720d-4cb7-a1da-cece1197a475-kube-api-access-bf2d8\") pod \"ingress-canary-s7mrr\" (UID: \"294058ff-720d-4cb7-a1da-cece1197a475\") " pod="openshift-ingress-canary/ingress-canary-s7mrr" Sep 09 08:40:07.787810 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.787697 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-metrics-tls\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.787810 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.787735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-tmp-dir\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.787810 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.787748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw2pm\" (UniqueName: \"kubernetes.io/projected/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-kube-api-access-hw2pm\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.788160 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.788137 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-tmp-dir\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.788339 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.788318 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-config-volume\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.791351 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.791334 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-metrics-tls\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.791506 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.791492 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/294058ff-720d-4cb7-a1da-cece1197a475-cert\") pod \"ingress-canary-s7mrr\" (UID: \"294058ff-720d-4cb7-a1da-cece1197a475\") " pod="openshift-ingress-canary/ingress-canary-s7mrr" Sep 09 08:40:07.797001 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.796957 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf2d8\" (UniqueName: \"kubernetes.io/projected/294058ff-720d-4cb7-a1da-cece1197a475-kube-api-access-bf2d8\") pod \"ingress-canary-s7mrr\" (UID: \"294058ff-720d-4cb7-a1da-cece1197a475\") " pod="openshift-ingress-canary/ingress-canary-s7mrr" Sep 09 08:40:07.797103 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.797087 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw2pm\" (UniqueName: \"kubernetes.io/projected/6bb6dfa7-85ee-487b-bec3-e3e99c71a526-kube-api-access-hw2pm\") pod \"dns-default-c9gmd\" (UID: \"6bb6dfa7-85ee-487b-bec3-e3e99c71a526\") " pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.846067 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.846035 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:07.853817 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:07.853795 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s7mrr" Sep 09 08:40:08.127024 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:08.126924 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s7mrr"] Sep 09 08:40:08.130464 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:08.130429 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c9gmd"] Sep 09 08:40:08.131096 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:08.131066 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294058ff_720d_4cb7_a1da_cece1197a475.slice/crio-168a98f8292de799f6839424b67a7fdb5b5bd80b66631fea0be5e5c348fff783 WatchSource:0}: Error finding container 168a98f8292de799f6839424b67a7fdb5b5bd80b66631fea0be5e5c348fff783: Status 404 returned error can't find the container with id 168a98f8292de799f6839424b67a7fdb5b5bd80b66631fea0be5e5c348fff783 Sep 09 08:40:08.135631 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:08.135606 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb6dfa7_85ee_487b_bec3_e3e99c71a526.slice/crio-f1da76d2e99658d0894dd08bb0aa614b244e7c905fb77761ddd2ac558c0e75ce WatchSource:0}: Error finding container f1da76d2e99658d0894dd08bb0aa614b244e7c905fb77761ddd2ac558c0e75ce: Status 404 returned error can't find the container with id f1da76d2e99658d0894dd08bb0aa614b244e7c905fb77761ddd2ac558c0e75ce Sep 09 08:40:08.897395 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:08.897351 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s7mrr" event={"ID":"294058ff-720d-4cb7-a1da-cece1197a475","Type":"ContainerStarted","Data":"168a98f8292de799f6839424b67a7fdb5b5bd80b66631fea0be5e5c348fff783"} Sep 09 08:40:08.898715 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:08.898686 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9gmd" event={"ID":"6bb6dfa7-85ee-487b-bec3-e3e99c71a526","Type":"ContainerStarted","Data":"f1da76d2e99658d0894dd08bb0aa614b244e7c905fb77761ddd2ac558c0e75ce"} Sep 09 08:40:09.903566 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:09.903460 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9gmd" event={"ID":"6bb6dfa7-85ee-487b-bec3-e3e99c71a526","Type":"ContainerStarted","Data":"49aa0d8ddc9891be23414727b201edb7e89fd228e5d24afc525d699ba350ccf0"} Sep 09 08:40:10.908099 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:10.908012 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s7mrr" event={"ID":"294058ff-720d-4cb7-a1da-cece1197a475","Type":"ContainerStarted","Data":"f1fc2642cb6d44d5a1ece26f827cb7c7a50e0eef6cfa2974e2869d011f12520d"} Sep 09 08:40:10.909776 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:10.909752 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9gmd" event={"ID":"6bb6dfa7-85ee-487b-bec3-e3e99c71a526","Type":"ContainerStarted","Data":"b5a79ab7feca97e961c89c9eab63547188b4ba702fe9d4f420c95356de354109"} Sep 09 08:40:10.909905 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:10.909879 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:10.924542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:10.924502 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s7mrr" podStartSLOduration=1.475984146 podStartE2EDuration="3.924489737s" totalImagesPullingTime="2.448505591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:10.923334106 +0000 UTC m=+54.756697665" watchObservedRunningTime="2026-09-09 08:40:10.924489737 +0000 UTC m=+54.757853286" Sep 09 08:40:10.939605 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:10.939562 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c9gmd" podStartSLOduration=2.360041927 podStartE2EDuration="3.939547786s" totalImagesPullingTime="1.579505859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:10.93907047 +0000 UTC m=+54.772434054" watchObservedRunningTime="2026-09-09 08:40:10.939547786 +0000 UTC m=+54.772911345" Sep 09 08:40:12.670545 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.670507 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7btxn"] Sep 09 08:40:12.705521 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.705488 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7btxn"] Sep 09 08:40:12.705673 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.705602 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.707867 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.707845 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 08:40:12.708019 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.707934 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 08:40:12.708128 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.708113 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 08:40:12.709140 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.709121 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9cg75\"" Sep 09 08:40:12.709240 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.709185 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 08:40:12.718643 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.718620 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.718757 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.718677 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12fd7ed4-407d-4fa2-854b-a483c8d81072-data-volume\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.718757 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.718704 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12fd7ed4-407d-4fa2-854b-a483c8d81072-crio-socket\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.718757 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.718731 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.718757 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.718752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx8b5\" (UniqueName: \"kubernetes.io/projected/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-api-access-nx8b5\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.819774 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.819733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.819961 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.819789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12fd7ed4-407d-4fa2-854b-a483c8d81072-data-volume\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.819961 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.819807 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12fd7ed4-407d-4fa2-854b-a483c8d81072-crio-socket\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.819961 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.819825 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.819961 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.819842 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx8b5\" (UniqueName: \"kubernetes.io/projected/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-api-access-nx8b5\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.819961 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:12.819856 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 08:40:12.819961 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.819944 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12fd7ed4-407d-4fa2-854b-a483c8d81072-crio-socket\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.820228 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:12.819954 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm podName:12fd7ed4-407d-4fa2-854b-a483c8d81072 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:13.319926611 +0000 UTC m=+57.153290149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7btxn" (UID: "12fd7ed4-407d-4fa2-854b-a483c8d81072") : configmap "kube-rbac-proxy" not found Sep 09 08:40:12.820228 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:12.819995 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 08:40:12.820228 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:12.820061 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls podName:12fd7ed4-407d-4fa2-854b-a483c8d81072 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:13.320041936 +0000 UTC m=+57.153405472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7btxn" (UID: "12fd7ed4-407d-4fa2-854b-a483c8d81072") : secret "insights-runtime-extractor-tls" not found Sep 09 08:40:12.820330 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.820227 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12fd7ed4-407d-4fa2-854b-a483c8d81072-data-volume\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:12.828707 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:12.828683 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx8b5\" (UniqueName: \"kubernetes.io/projected/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-api-access-nx8b5\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:13.322830 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:13.322783 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:13.323065 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:13.322863 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:13.323065 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:13.322940 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 08:40:13.323065 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:13.322958 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 08:40:13.323065 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:13.323037 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm podName:12fd7ed4-407d-4fa2-854b-a483c8d81072 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:14.323018567 +0000 UTC m=+58.156382109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7btxn" (UID: "12fd7ed4-407d-4fa2-854b-a483c8d81072") : configmap "kube-rbac-proxy" not found Sep 09 08:40:13.323065 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:13.323056 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls podName:12fd7ed4-407d-4fa2-854b-a483c8d81072 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:14.323047018 +0000 UTC m=+58.156410556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7btxn" (UID: "12fd7ed4-407d-4fa2-854b-a483c8d81072") : secret "insights-runtime-extractor-tls" not found Sep 09 08:40:13.668358 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:13.668283 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c9gmd_6bb6dfa7-85ee-487b-bec3-e3e99c71a526/dns/0.log" Sep 09 08:40:13.853896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:13.853873 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c9gmd_6bb6dfa7-85ee-487b-bec3-e3e99c71a526/kube-rbac-proxy/0.log" Sep 09 08:40:13.991202 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:13.991121 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw"] Sep 09 08:40:13.992984 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:13.992958 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" Sep 09 08:40:13.995648 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:13.995630 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jmr6b\"" Sep 09 08:40:13.996853 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:13.996830 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 09 08:40:14.005257 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.005236 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw"] Sep 09 08:40:14.026608 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.026588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a957469c-2738-4ce2-b169-39432f7394f0-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-c7qkw\" (UID: \"a957469c-2738-4ce2-b169-39432f7394f0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" Sep 09 08:40:14.127212 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.127176 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a957469c-2738-4ce2-b169-39432f7394f0-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-c7qkw\" (UID: \"a957469c-2738-4ce2-b169-39432f7394f0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" Sep 09 08:40:14.127380 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:14.127290 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 09 08:40:14.127380 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:14.127341 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a957469c-2738-4ce2-b169-39432f7394f0-tls-certificates podName:a957469c-2738-4ce2-b169-39432f7394f0 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:14.627328864 +0000 UTC m=+58.460692401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a957469c-2738-4ce2-b169-39432f7394f0-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-c7qkw" (UID: "a957469c-2738-4ce2-b169-39432f7394f0") : secret "prometheus-operator-admission-webhook-tls" not found Sep 09 08:40:14.328166 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.328132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:14.328328 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.328203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:14.328328 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:14.328285 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 08:40:14.328415 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:14.328349 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls podName:12fd7ed4-407d-4fa2-854b-a483c8d81072 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:16.3283356 +0000 UTC m=+60.161699136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7btxn" (UID: "12fd7ed4-407d-4fa2-854b-a483c8d81072") : secret "insights-runtime-extractor-tls" not found Sep 09 08:40:14.328415 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:14.328289 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 08:40:14.328488 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:14.328432 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm podName:12fd7ed4-407d-4fa2-854b-a483c8d81072 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:16.328420388 +0000 UTC m=+60.161783924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7btxn" (UID: "12fd7ed4-407d-4fa2-854b-a483c8d81072") : configmap "kube-rbac-proxy" not found Sep 09 08:40:14.630098 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.630007 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a957469c-2738-4ce2-b169-39432f7394f0-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-c7qkw\" (UID: \"a957469c-2738-4ce2-b169-39432f7394f0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" Sep 09 08:40:14.632460 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.632435 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a957469c-2738-4ce2-b169-39432f7394f0-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-c7qkw\" (UID: \"a957469c-2738-4ce2-b169-39432f7394f0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" Sep 09 08:40:14.654190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.654164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lcbk4_7420c89d-6441-4350-9c6a-6a5b3fd8c2ca/dns-node-resolver/0.log" Sep 09 08:40:14.901234 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:14.901157 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" Sep 09 08:40:15.166707 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:15.166631 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw"] Sep 09 08:40:15.169611 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:15.169586 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda957469c_2738_4ce2_b169_39432f7394f0.slice/crio-4f8baf8021c19ef0cbf4cd88163e509a3d3a8afc5f5063e0e42bc8fe460112af WatchSource:0}: Error finding container 4f8baf8021c19ef0cbf4cd88163e509a3d3a8afc5f5063e0e42bc8fe460112af: Status 404 returned error can't find the container with id 4f8baf8021c19ef0cbf4cd88163e509a3d3a8afc5f5063e0e42bc8fe460112af Sep 09 08:40:15.654573 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:15.654544 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jmn4t_333ce92a-033d-45bc-bbfc-78ea69ff251c/node-ca/0.log" Sep 09 08:40:15.845592 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:15.845562 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jw7m5" Sep 09 08:40:15.923702 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:15.923618 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" event={"ID":"a957469c-2738-4ce2-b169-39432f7394f0","Type":"ContainerStarted","Data":"4f8baf8021c19ef0cbf4cd88163e509a3d3a8afc5f5063e0e42bc8fe460112af"} Sep 09 08:40:16.341209 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.341173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:16.341393 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.341236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:16.341393 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:16.341303 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 08:40:16.341393 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:16.341309 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 08:40:16.341393 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:16.341361 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls podName:12fd7ed4-407d-4fa2-854b-a483c8d81072 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:20.341346033 +0000 UTC m=+64.174709570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7btxn" (UID: "12fd7ed4-407d-4fa2-854b-a483c8d81072") : secret "insights-runtime-extractor-tls" not found Sep 09 08:40:16.341393 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:16.341375 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm podName:12fd7ed4-407d-4fa2-854b-a483c8d81072 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:20.341369546 +0000 UTC m=+64.174733083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7btxn" (UID: "12fd7ed4-407d-4fa2-854b-a483c8d81072") : configmap "kube-rbac-proxy" not found Sep 09 08:40:16.581420 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.581343 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-4rmzn"] Sep 09 08:40:16.583413 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.583398 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-4rmzn" Sep 09 08:40:16.585641 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.585624 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kf245\"" Sep 09 08:40:16.585932 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.585918 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 09 08:40:16.586035 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.586016 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 09 08:40:16.592218 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.592197 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-4rmzn"] Sep 09 08:40:16.643136 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.643104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twrpk\" (UniqueName: \"kubernetes.io/projected/743d622f-3fd6-4d52-be02-fe88084a7cc2-kube-api-access-twrpk\") pod \"downloads-85d98c998f-4rmzn\" (UID: \"743d622f-3fd6-4d52-be02-fe88084a7cc2\") " pod="openshift-console/downloads-85d98c998f-4rmzn" Sep 09 08:40:16.654687 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.654665 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s7mrr_294058ff-720d-4cb7-a1da-cece1197a475/serve-healthcheck-canary/0.log" Sep 09 08:40:16.744025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.743957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twrpk\" (UniqueName: \"kubernetes.io/projected/743d622f-3fd6-4d52-be02-fe88084a7cc2-kube-api-access-twrpk\") pod \"downloads-85d98c998f-4rmzn\" (UID: \"743d622f-3fd6-4d52-be02-fe88084a7cc2\") " pod="openshift-console/downloads-85d98c998f-4rmzn" Sep 09 08:40:16.751708 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.751676 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twrpk\" (UniqueName: \"kubernetes.io/projected/743d622f-3fd6-4d52-be02-fe88084a7cc2-kube-api-access-twrpk\") pod \"downloads-85d98c998f-4rmzn\" (UID: \"743d622f-3fd6-4d52-be02-fe88084a7cc2\") " pod="openshift-console/downloads-85d98c998f-4rmzn" Sep 09 08:40:16.892404 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.892318 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-4rmzn" Sep 09 08:40:16.927885 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.927847 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" event={"ID":"a957469c-2738-4ce2-b169-39432f7394f0","Type":"ContainerStarted","Data":"53f260fde225591e9715ad371b7cb00544ec7590c49107e696dad606eb435864"} Sep 09 08:40:16.928324 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.928053 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" Sep 09 08:40:16.933185 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.933165 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" Sep 09 08:40:16.944164 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:16.944110 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-c7qkw" podStartSLOduration=2.866223933 podStartE2EDuration="3.944087092s" totalImagesPullingTime="1.077863159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:16.943314559 +0000 UTC m=+60.776678122" watchObservedRunningTime="2026-09-09 08:40:16.944087092 +0000 UTC m=+60.777450648" Sep 09 08:40:17.159309 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:17.159186 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-4rmzn"] Sep 09 08:40:17.161672 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:17.161641 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod743d622f_3fd6_4d52_be02_fe88084a7cc2.slice/crio-f35a80c756374097281b0faff8ed72464a5865356515a4591d96ae48ac9c2842 WatchSource:0}: Error finding container f35a80c756374097281b0faff8ed72464a5865356515a4591d96ae48ac9c2842: Status 404 returned error can't find the container with id f35a80c756374097281b0faff8ed72464a5865356515a4591d96ae48ac9c2842 Sep 09 08:40:17.931754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:17.931715 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-4rmzn" event={"ID":"743d622f-3fd6-4d52-be02-fe88084a7cc2","Type":"ContainerStarted","Data":"f35a80c756374097281b0faff8ed72464a5865356515a4591d96ae48ac9c2842"} Sep 09 08:40:18.053951 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.053918 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2rt5x"] Sep 09 08:40:18.055889 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.055867 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.058271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.058247 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 09 08:40:18.058420 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.058396 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nqlhq\"" Sep 09 08:40:18.059787 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.059616 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 08:40:18.059787 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.059680 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 09 08:40:18.059787 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.059716 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 08:40:18.059787 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.059687 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 08:40:18.066087 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.066066 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2rt5x"] Sep 09 08:40:18.155196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.155150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fee83f8d-c796-4c40-a05d-bb5eb71c4531-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.155196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.155199 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fee83f8d-c796-4c40-a05d-bb5eb71c4531-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.155419 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.155308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fee83f8d-c796-4c40-a05d-bb5eb71c4531-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.155419 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.155365 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-989tw\" (UniqueName: \"kubernetes.io/projected/fee83f8d-c796-4c40-a05d-bb5eb71c4531-kube-api-access-989tw\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.255788 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.255756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fee83f8d-c796-4c40-a05d-bb5eb71c4531-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.256025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.255796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fee83f8d-c796-4c40-a05d-bb5eb71c4531-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.256025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.255852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fee83f8d-c796-4c40-a05d-bb5eb71c4531-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.256025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.255882 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-989tw\" (UniqueName: \"kubernetes.io/projected/fee83f8d-c796-4c40-a05d-bb5eb71c4531-kube-api-access-989tw\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.256739 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.256714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fee83f8d-c796-4c40-a05d-bb5eb71c4531-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.258613 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.258586 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fee83f8d-c796-4c40-a05d-bb5eb71c4531-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.258865 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.258841 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fee83f8d-c796-4c40-a05d-bb5eb71c4531-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.264990 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.264946 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-989tw\" (UniqueName: \"kubernetes.io/projected/fee83f8d-c796-4c40-a05d-bb5eb71c4531-kube-api-access-989tw\") pod \"prometheus-operator-6f4946db56-2rt5x\" (UID: \"fee83f8d-c796-4c40-a05d-bb5eb71c4531\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.367072 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.367037 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" Sep 09 08:40:18.644370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.644288 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2rt5x"] Sep 09 08:40:18.647577 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:18.647534 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfee83f8d_c796_4c40_a05d_bb5eb71c4531.slice/crio-1d33ce11d3f95ad1e2ba0d03406ad6b113c88c0ebb31856c0782315ecbbe55c5 WatchSource:0}: Error finding container 1d33ce11d3f95ad1e2ba0d03406ad6b113c88c0ebb31856c0782315ecbbe55c5: Status 404 returned error can't find the container with id 1d33ce11d3f95ad1e2ba0d03406ad6b113c88c0ebb31856c0782315ecbbe55c5 Sep 09 08:40:18.935428 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:18.935344 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" event={"ID":"fee83f8d-c796-4c40-a05d-bb5eb71c4531","Type":"ContainerStarted","Data":"1d33ce11d3f95ad1e2ba0d03406ad6b113c88c0ebb31856c0782315ecbbe55c5"} Sep 09 08:40:20.376531 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.376497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:20.377006 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.376582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:20.377534 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.377511 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12fd7ed4-407d-4fa2-854b-a483c8d81072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:20.379506 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.379486 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12fd7ed4-407d-4fa2-854b-a483c8d81072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7btxn\" (UID: \"12fd7ed4-407d-4fa2-854b-a483c8d81072\") " pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:20.518755 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.518708 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9cg75\"" Sep 09 08:40:20.526938 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.526910 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7btxn" Sep 09 08:40:20.819332 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.819256 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7btxn"] Sep 09 08:40:20.822542 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:20.822511 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fd7ed4_407d_4fa2_854b_a483c8d81072.slice/crio-995162a20ec1909da8268f9fefb6d27651dd06561d780162af63fcdc60b63f10 WatchSource:0}: Error finding container 995162a20ec1909da8268f9fefb6d27651dd06561d780162af63fcdc60b63f10: Status 404 returned error can't find the container with id 995162a20ec1909da8268f9fefb6d27651dd06561d780162af63fcdc60b63f10 Sep 09 08:40:20.915116 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.915091 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c9gmd" Sep 09 08:40:20.945731 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.945593 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7btxn" event={"ID":"12fd7ed4-407d-4fa2-854b-a483c8d81072","Type":"ContainerStarted","Data":"a739bf85135528e615e0e0b041d229c2a7e1fe48490b50fca0bb8bb5a9034965"} Sep 09 08:40:20.945731 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.945694 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7btxn" event={"ID":"12fd7ed4-407d-4fa2-854b-a483c8d81072","Type":"ContainerStarted","Data":"995162a20ec1909da8268f9fefb6d27651dd06561d780162af63fcdc60b63f10"} Sep 09 08:40:20.947964 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.947936 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" event={"ID":"fee83f8d-c796-4c40-a05d-bb5eb71c4531","Type":"ContainerStarted","Data":"66f26e3242095c20160105c35cc3afdf5d2b91128f57cd3bb673ebe4637ccc4a"} Sep 09 08:40:20.948098 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.948005 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" event={"ID":"fee83f8d-c796-4c40-a05d-bb5eb71c4531","Type":"ContainerStarted","Data":"26316824cb3cde79baa6f914b1d3eb57db40e30bbf4e6a0cc9f1fcc68bbc6827"} Sep 09 08:40:20.965655 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:20.965317 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-2rt5x" podStartSLOduration=1.442292733 podStartE2EDuration="2.96529792s" totalImagesPullingTime="1.523005187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:20.964176462 +0000 UTC m=+64.797540043" watchObservedRunningTime="2026-09-09 08:40:20.96529792 +0000 UTC m=+64.798661480" Sep 09 08:40:21.954575 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:21.954484 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7btxn" event={"ID":"12fd7ed4-407d-4fa2-854b-a483c8d81072","Type":"ContainerStarted","Data":"4473b83295b7ffc9fdb84360de82dddcdfcfc98ac67ccb32fe95b601568feccf"} Sep 09 08:40:22.392685 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.392643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:40:22.392859 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.392707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:40:22.392859 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.392756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:40:22.395484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.395457 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 08:40:22.395484 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.395470 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 08:40:22.396678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.396651 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 08:40:22.406003 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.405801 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 08:40:22.406596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.406544 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e9b855f-9ef8-4c2f-903d-492a4119d176-original-pull-secret\") pod \"global-pull-secret-syncer-c25ds\" (UID: \"6e9b855f-9ef8-4c2f-903d-492a4119d176\") " pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:40:22.406955 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.406935 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4992530-e8b6-4ebd-bc09-04038870dfbf-metrics-certs\") pod \"network-metrics-daemon-wqxmh\" (UID: \"a4992530-e8b6-4ebd-bc09-04038870dfbf\") " pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:40:22.416732 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.416700 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t9k6\" (UniqueName: \"kubernetes.io/projected/dbeeae9e-b538-462d-a818-c8914cfdc325-kube-api-access-8t9k6\") pod \"network-check-target-9w6g4\" (UID: \"dbeeae9e-b538-462d-a818-c8914cfdc325\") " pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:40:22.428399 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.428374 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q"] Sep 09 08:40:22.430986 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.430892 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.433851 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.433825 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 09 08:40:22.433953 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.433878 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-sshqw\"" Sep 09 08:40:22.433953 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.433878 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 09 08:40:22.443807 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.443749 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q"] Sep 09 08:40:22.456434 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.456406 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4"] Sep 09 08:40:22.458640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.458617 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.461562 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.461541 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 09 08:40:22.461662 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.461570 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-x2fct\"" Sep 09 08:40:22.461662 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.461627 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 09 08:40:22.461838 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.461683 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 09 08:40:22.472052 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.472028 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4"] Sep 09 08:40:22.474095 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.474073 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kp2sc"] Sep 09 08:40:22.476531 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.476513 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.478754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.478735 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nngdh\"" Sep 09 08:40:22.478861 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.478821 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 08:40:22.478962 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.478942 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 08:40:22.479899 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.479396 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 08:40:22.493734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493626 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.493734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493667 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-textfile\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.493734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493698 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c61504b7-2791-49ff-9631-c5ef7b12c6a8-metrics-client-ca\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.493952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/740bc940-3f52-4ad9-8a98-ad998acdd9a0-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.493952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493799 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-root\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.493952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.493952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.493952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.493952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493948 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.494267 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.493991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-tls\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.494267 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494059 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52n7q\" (UniqueName: \"kubernetes.io/projected/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-api-access-52n7q\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.494267 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.494267 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494119 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-sys\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.494267 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znnr4\" (UniqueName: \"kubernetes.io/projected/c61504b7-2791-49ff-9631-c5ef7b12c6a8-kube-api-access-znnr4\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.494267 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.494267 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6ppq\" (UniqueName: \"kubernetes.io/projected/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-kube-api-access-s6ppq\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.494551 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494281 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-wtmp\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.494551 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.494551 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.494340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/740bc940-3f52-4ad9-8a98-ad998acdd9a0-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.594804 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/740bc940-3f52-4ad9-8a98-ad998acdd9a0-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.594865 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.594901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-textfile\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.594933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c61504b7-2791-49ff-9631-c5ef7b12c6a8-metrics-client-ca\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.594959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/740bc940-3f52-4ad9-8a98-ad998acdd9a0-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595026 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-root\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595063 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595198 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-tls\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52n7q\" (UniqueName: \"kubernetes.io/projected/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-api-access-52n7q\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595277 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-sys\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znnr4\" (UniqueName: \"kubernetes.io/projected/c61504b7-2791-49ff-9631-c5ef7b12c6a8-kube-api-access-znnr4\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.595398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595347 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.596307 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595381 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6ppq\" (UniqueName: \"kubernetes.io/projected/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-kube-api-access-s6ppq\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.596307 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595404 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-wtmp\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.596307 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.596307 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595665 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c61504b7-2791-49ff-9631-c5ef7b12c6a8-metrics-client-ca\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.596307 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595664 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/740bc940-3f52-4ad9-8a98-ad998acdd9a0-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.596307 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:22.595789 2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 09 08:40:22.596307 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:22.595846 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-tls podName:c61504b7-2791-49ff-9631-c5ef7b12c6a8 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:23.095826109 +0000 UTC m=+66.929189651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-tls") pod "node-exporter-kp2sc" (UID: "c61504b7-2791-49ff-9631-c5ef7b12c6a8") : secret "node-exporter-tls" not found Sep 09 08:40:22.596307 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.595887 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-textfile\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.596699 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:22.596405 2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 09 08:40:22.596699 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:22.596462 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-tls podName:1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:23.096441476 +0000 UTC m=+66.929805028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-4cg7q" (UID: "1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6") : secret "openshift-state-metrics-tls" not found Sep 09 08:40:22.596699 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:22.596461 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 09 08:40:22.596699 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.596513 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-sys\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.596699 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:22.596533 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-tls podName:740bc940-3f52-4ad9-8a98-ad998acdd9a0 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:23.096515374 +0000 UTC m=+66.929878915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-kfdk4" (UID: "740bc940-3f52-4ad9-8a98-ad998acdd9a0") : secret "kube-state-metrics-tls" not found Sep 09 08:40:22.596699 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.596591 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-root\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.596699 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.596647 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-wtmp\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.597181 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.596808 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.597181 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.596865 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/740bc940-3f52-4ad9-8a98-ad998acdd9a0-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.597181 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.597156 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.597425 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.597401 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.599187 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.599159 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.599521 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.599497 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.600271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.600197 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.609025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.607536 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znnr4\" (UniqueName: \"kubernetes.io/projected/c61504b7-2791-49ff-9631-c5ef7b12c6a8-kube-api-access-znnr4\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:22.609025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.607911 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52n7q\" (UniqueName: \"kubernetes.io/projected/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-api-access-52n7q\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:22.609025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.608344 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6ppq\" (UniqueName: \"kubernetes.io/projected/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-kube-api-access-s6ppq\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:22.637290 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.637257 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c25ds" Sep 09 08:40:22.644895 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.644837 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4bf2n\"" Sep 09 08:40:22.650129 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.650107 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dtwcb\"" Sep 09 08:40:22.653186 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.653164 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wqxmh" Sep 09 08:40:22.657986 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:22.657957 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:40:23.101286 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.100528 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-tls\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:23.101286 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.100568 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:23.101286 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.100590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:23.103889 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.103795 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4cg7q\" (UID: \"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:23.105132 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.105085 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/740bc940-3f52-4ad9-8a98-ad998acdd9a0-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-kfdk4\" (UID: \"740bc940-3f52-4ad9-8a98-ad998acdd9a0\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:23.105410 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.105234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c61504b7-2791-49ff-9631-c5ef7b12c6a8-node-exporter-tls\") pod \"node-exporter-kp2sc\" (UID: \"c61504b7-2791-49ff-9631-c5ef7b12c6a8\") " pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:23.343285 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.343208 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" Sep 09 08:40:23.356286 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.356235 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wqxmh"] Sep 09 08:40:23.358150 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.358089 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c25ds"] Sep 09 08:40:23.359656 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:23.359623 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4992530_e8b6_4ebd_bc09_04038870dfbf.slice/crio-c08d72d78b2b633c66baac4a08b3b708bf16ae0512e9fa771a79c3e0bbe8a65b WatchSource:0}: Error finding container c08d72d78b2b633c66baac4a08b3b708bf16ae0512e9fa771a79c3e0bbe8a65b: Status 404 returned error can't find the container with id c08d72d78b2b633c66baac4a08b3b708bf16ae0512e9fa771a79c3e0bbe8a65b Sep 09 08:40:23.362798 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.362738 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9w6g4"] Sep 09 08:40:23.365630 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:23.365600 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9b855f_9ef8_4c2f_903d_492a4119d176.slice/crio-85bcf510c52fc86be16a7a448f3860f6a950cd4ad6b455e7abb1939373b72575 WatchSource:0}: Error finding container 85bcf510c52fc86be16a7a448f3860f6a950cd4ad6b455e7abb1939373b72575: Status 404 returned error can't find the container with id 85bcf510c52fc86be16a7a448f3860f6a950cd4ad6b455e7abb1939373b72575 Sep 09 08:40:23.367566 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:23.367483 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbeeae9e_b538_462d_a818_c8914cfdc325.slice/crio-279193db78fe79a15fca84ad8398f5b7e686d8c8a1a2ee88a61fe42eaa2195fc WatchSource:0}: Error finding container 279193db78fe79a15fca84ad8398f5b7e686d8c8a1a2ee88a61fe42eaa2195fc: Status 404 returned error can't find the container with id 279193db78fe79a15fca84ad8398f5b7e686d8c8a1a2ee88a61fe42eaa2195fc Sep 09 08:40:23.369455 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.369426 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" Sep 09 08:40:23.387345 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.387248 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kp2sc" Sep 09 08:40:23.404018 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:23.403953 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61504b7_2791_49ff_9631_c5ef7b12c6a8.slice/crio-2ada423c477de0f8da62ae0537672d062402cbf1215f543fc14d67e715617747 WatchSource:0}: Error finding container 2ada423c477de0f8da62ae0537672d062402cbf1215f543fc14d67e715617747: Status 404 returned error can't find the container with id 2ada423c477de0f8da62ae0537672d062402cbf1215f543fc14d67e715617747 Sep 09 08:40:23.528348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.528309 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:40:23.531805 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.531776 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.534636 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.534606 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 08:40:23.534636 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.534626 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 08:40:23.534849 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.534606 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 08:40:23.535386 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.535104 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 08:40:23.535386 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.535165 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 08:40:23.535386 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.535188 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 08:40:23.535386 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.535238 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 08:40:23.535386 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.535197 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 08:40:23.535386 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.535328 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hjqx8\"" Sep 09 08:40:23.535789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.535502 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 08:40:23.546616 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.546593 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:40:23.605804 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.605718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.605804 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.605769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdm88\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-kube-api-access-wdm88\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.605804 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.605797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.605864 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.605902 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.605931 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.605951 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.605967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-out\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.606006 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.606023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.606043 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-volume\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.606058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-web-config\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.606075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.606074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.635676 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.635636 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q"] Sep 09 08:40:23.639065 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:23.639035 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3b1ffe_3f4e_4ff6_b892_624a0f15e3a6.slice/crio-b9d4092ea005e213dcc1a95fa60d160f7769e0c940830ebda7ae3e3dff4f22d5 WatchSource:0}: Error finding container b9d4092ea005e213dcc1a95fa60d160f7769e0c940830ebda7ae3e3dff4f22d5: Status 404 returned error can't find the container with id b9d4092ea005e213dcc1a95fa60d160f7769e0c940830ebda7ae3e3dff4f22d5 Sep 09 08:40:23.683759 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.683724 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4"] Sep 09 08:40:23.692052 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:23.692028 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740bc940_3f52_4ad9_8a98_ad998acdd9a0.slice/crio-197247dc9a769143664157053f5ecc6859307e461613e2617d1c8b59943ee315 WatchSource:0}: Error finding container 197247dc9a769143664157053f5ecc6859307e461613e2617d1c8b59943ee315: Status 404 returned error can't find the container with id 197247dc9a769143664157053f5ecc6859307e461613e2617d1c8b59943ee315 Sep 09 08:40:23.707010 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.706965 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707103 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707026 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-out\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707103 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707055 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707103 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707269 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707113 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-volume\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707269 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707137 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-web-config\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707269 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707163 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707269 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707215 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707460 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdm88\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-kube-api-access-wdm88\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707460 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707460 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707460 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707460 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.707680 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.707417 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.708686 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:23.707849 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle podName:9ddd96dc-9c23-44d0-909b-62c0551bdff6 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:24.207826965 +0000 UTC m=+68.041190504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6") : configmap references non-existent config key: ca-bundle.crt Sep 09 08:40:23.708686 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:23.708360 2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 09 08:40:23.708686 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:23.708413 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls podName:9ddd96dc-9c23-44d0-909b-62c0551bdff6 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:24.208395355 +0000 UTC m=+68.041758909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6") : secret "alertmanager-main-tls" not found Sep 09 08:40:23.708686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.708617 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.712758 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.712714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-out\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.713442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.713402 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-web-config\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.713683 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.713585 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-volume\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.713776 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.713699 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.714071 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.714032 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.714174 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.714084 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.714703 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.714680 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.716491 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.716468 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.728662 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.728615 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdm88\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-kube-api-access-wdm88\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:23.968429 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.968342 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kp2sc" event={"ID":"c61504b7-2791-49ff-9631-c5ef7b12c6a8","Type":"ContainerStarted","Data":"2ada423c477de0f8da62ae0537672d062402cbf1215f543fc14d67e715617747"} Sep 09 08:40:23.972628 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.972482 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" event={"ID":"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6","Type":"ContainerStarted","Data":"4ecb7431e6f7e28d226bb24d3534488902d4c618d97a857b46015da331d04615"} Sep 09 08:40:23.972628 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.972543 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" event={"ID":"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6","Type":"ContainerStarted","Data":"02f7dd20036e19a1865e746f48794f5e59fdb41ec17a82f1184c2f518661eb95"} Sep 09 08:40:23.972628 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.972573 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" event={"ID":"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6","Type":"ContainerStarted","Data":"b9d4092ea005e213dcc1a95fa60d160f7769e0c940830ebda7ae3e3dff4f22d5"} Sep 09 08:40:23.974471 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.974399 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c25ds" event={"ID":"6e9b855f-9ef8-4c2f-903d-492a4119d176","Type":"ContainerStarted","Data":"85bcf510c52fc86be16a7a448f3860f6a950cd4ad6b455e7abb1939373b72575"} Sep 09 08:40:23.980068 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.979191 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7btxn" event={"ID":"12fd7ed4-407d-4fa2-854b-a483c8d81072","Type":"ContainerStarted","Data":"b1dc9f11647217a24d46acc65ff32c542ac74855c0709e1184197d1026b96b4b"} Sep 09 08:40:23.981666 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.981588 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" event={"ID":"740bc940-3f52-4ad9-8a98-ad998acdd9a0","Type":"ContainerStarted","Data":"197247dc9a769143664157053f5ecc6859307e461613e2617d1c8b59943ee315"} Sep 09 08:40:23.994057 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.993927 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9w6g4" event={"ID":"dbeeae9e-b538-462d-a818-c8914cfdc325","Type":"ContainerStarted","Data":"279193db78fe79a15fca84ad8398f5b7e686d8c8a1a2ee88a61fe42eaa2195fc"} Sep 09 08:40:24.000004 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:23.997633 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wqxmh" event={"ID":"a4992530-e8b6-4ebd-bc09-04038870dfbf","Type":"ContainerStarted","Data":"c08d72d78b2b633c66baac4a08b3b708bf16ae0512e9fa771a79c3e0bbe8a65b"} Sep 09 08:40:24.011640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:24.011590 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7btxn" podStartSLOduration=9.882757433 podStartE2EDuration="12.011572452s" totalImagesPullingTime="2.128815019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:40:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:24.010607204 +0000 UTC m=+67.843970766" watchObservedRunningTime="2026-09-09 08:40:24.011572452 +0000 UTC m=+67.844936015" Sep 09 08:40:24.213772 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:24.213239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:24.213772 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:24.213292 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:24.215249 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:24.215172 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:24.218548 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:24.218468 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:24.444137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:24.444102 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:40:25.417226 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.415335 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s"] Sep 09 08:40:25.420045 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.418990 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.422908 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.422151 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1r1jf1fnnrijg\"" Sep 09 08:40:25.422908 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.422424 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 09 08:40:25.422908 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.422597 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 09 08:40:25.422908 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.422754 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 09 08:40:25.424310 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.423478 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 09 08:40:25.424310 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.423595 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zxf4z\"" Sep 09 08:40:25.424310 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.423841 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 09 08:40:25.431356 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.430307 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s"] Sep 09 08:40:25.527469 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.527422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.527639 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.527493 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-grpc-tls\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.527639 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.527560 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.527639 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.527597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-tls\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.527639 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.527629 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.527906 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.527667 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.527906 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.527712 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-metrics-client-ca\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.527906 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.527745 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztwhp\" (UniqueName: \"kubernetes.io/projected/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-kube-api-access-ztwhp\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.629476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.628314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.629476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.628367 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-tls\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.629476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.628401 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.629476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.628436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.629476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.628474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-metrics-client-ca\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.629476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.628508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztwhp\" (UniqueName: \"kubernetes.io/projected/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-kube-api-access-ztwhp\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.629476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.628536 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.629476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.628578 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-grpc-tls\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.630074 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.629879 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-metrics-client-ca\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.641468 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.641378 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.641468 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.641378 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.642406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.641720 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.642406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.641806 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-tls\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.642406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.642169 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-grpc-tls\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.642406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.642291 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.644388 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.644342 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztwhp\" (UniqueName: \"kubernetes.io/projected/dc7027bc-de24-49d0-bc2a-ff7bba8f75b5-kube-api-access-ztwhp\") pod \"thanos-querier-76bcd6d8d4-kbr4s\" (UID: \"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5\") " pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:25.734681 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:25.734597 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:26.385164 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.384829 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s"] Sep 09 08:40:26.388170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.388099 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:40:26.724944 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.724861 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b7ffb9bcb-9cl22"] Sep 09 08:40:26.730209 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.730177 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.735004 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.734959 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 09 08:40:26.735130 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.735039 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 09 08:40:26.735189 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.734968 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 09 08:40:26.735363 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.735316 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vt9hd\"" Sep 09 08:40:26.735453 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.735432 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 09 08:40:26.736775 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.736627 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 09 08:40:26.739921 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.739232 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7ffb9bcb-9cl22"] Sep 09 08:40:26.840137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.838792 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z"] Sep 09 08:40:26.840493 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.840446 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-console-config\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.840607 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.840498 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26nkt\" (UniqueName: \"kubernetes.io/projected/3f738416-635d-4718-bfda-5eb1c058aa8f-kube-api-access-26nkt\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.840607 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.840529 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-serving-cert\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.840607 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.840561 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-oauth-config\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.840607 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.840596 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-service-ca\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.840829 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.840644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-oauth-serving-cert\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.842340 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.842319 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:26.845334 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.845068 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 09 08:40:26.845334 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.845157 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 08:40:26.845334 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.845068 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 09 08:40:26.845554 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.845351 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9dj2b\"" Sep 09 08:40:26.845554 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.845433 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dj0a7h7dg78eb\"" Sep 09 08:40:26.845554 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.845440 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 09 08:40:26.853258 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.853210 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z"] Sep 09 08:40:26.941503 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-metrics-server-audit-profiles\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:26.941503 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-secret-metrics-server-client-certs\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941538 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-audit-log\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-console-config\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941618 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941639 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-client-ca-bundle\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26nkt\" (UniqueName: \"kubernetes.io/projected/3f738416-635d-4718-bfda-5eb1c058aa8f-kube-api-access-26nkt\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941673 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-serving-cert\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941688 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-secret-metrics-server-tls\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-oauth-config\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.941754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7txq\" (UniqueName: \"kubernetes.io/projected/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-kube-api-access-p7txq\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:26.942178 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-service-ca\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.942178 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.941833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-oauth-serving-cert\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.942468 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.942435 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-service-ca\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.942565 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.942438 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-console-config\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.942689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.942664 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-oauth-serving-cert\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.945009 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.944946 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-serving-cert\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.945472 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.945423 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-oauth-config\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:26.950236 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:26.950216 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26nkt\" (UniqueName: \"kubernetes.io/projected/3f738416-635d-4718-bfda-5eb1c058aa8f-kube-api-access-26nkt\") pod \"console-6b7ffb9bcb-9cl22\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:27.016615 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.016571 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" event={"ID":"1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6","Type":"ContainerStarted","Data":"3359ab2c9f9b55243d5979649a2d6be3ccc61d0079f7c894d1271cea0b3b486e"} Sep 09 08:40:27.019216 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.019047 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" event={"ID":"740bc940-3f52-4ad9-8a98-ad998acdd9a0","Type":"ContainerStarted","Data":"27dccbdb8c3bb5555e5a3b4650c93c9446c65e2ceab1b63eac48a6ff80a59192"} Sep 09 08:40:27.035471 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.035351 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4cg7q" podStartSLOduration=2.7428895410000003 podStartE2EDuration="5.035330167s" totalImagesPullingTime="2.292440626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:27.033781655 +0000 UTC m=+70.867145209" watchObservedRunningTime="2026-09-09 08:40:27.035330167 +0000 UTC m=+70.868693726" Sep 09 08:40:27.042485 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.042449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-metrics-server-audit-profiles\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.042627 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.042500 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-secret-metrics-server-client-certs\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.042627 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.042528 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-audit-log\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.042627 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.042579 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.042627 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.042604 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-client-ca-bundle\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.042839 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.042636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-secret-metrics-server-tls\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.042839 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.042675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7txq\" (UniqueName: \"kubernetes.io/projected/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-kube-api-access-p7txq\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.043205 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.043090 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-audit-log\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.043666 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.043641 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.044026 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.044002 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-metrics-server-audit-profiles\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.047233 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.047184 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-secret-metrics-server-client-certs\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.047670 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.047354 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-secret-metrics-server-tls\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.047767 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.047753 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-client-ca-bundle\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.052090 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.052067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:40:27.057636 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.057558 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7txq\" (UniqueName: \"kubernetes.io/projected/0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc-kube-api-access-p7txq\") pod \"metrics-server-7dcb6bdff4-5gj4z\" (UID: \"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc\") " pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.155790 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.155749 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:27.190434 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.190226 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l"] Sep 09 08:40:27.194834 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.194146 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:27.197137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.197105 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 09 08:40:27.197637 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.197457 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-799wz\"" Sep 09 08:40:27.202753 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.202729 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l"] Sep 09 08:40:27.245900 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.245852 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/170864f0-3e0b-4d2e-a971-2edad85e5df2-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nfs8l\" (UID: \"170864f0-3e0b-4d2e-a971-2edad85e5df2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:27.246078 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.246000 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc5wj\" (UniqueName: \"kubernetes.io/projected/170864f0-3e0b-4d2e-a971-2edad85e5df2-kube-api-access-gc5wj\") pod \"monitoring-plugin-85bbbbbf4-nfs8l\" (UID: \"170864f0-3e0b-4d2e-a971-2edad85e5df2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:27.346896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.346803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/170864f0-3e0b-4d2e-a971-2edad85e5df2-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nfs8l\" (UID: \"170864f0-3e0b-4d2e-a971-2edad85e5df2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:27.346896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.346890 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc5wj\" (UniqueName: \"kubernetes.io/projected/170864f0-3e0b-4d2e-a971-2edad85e5df2-kube-api-access-gc5wj\") pod \"monitoring-plugin-85bbbbbf4-nfs8l\" (UID: \"170864f0-3e0b-4d2e-a971-2edad85e5df2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:27.347137 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:27.346994 2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 09 08:40:27.347137 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:40:27.347074 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/170864f0-3e0b-4d2e-a971-2edad85e5df2-monitoring-plugin-cert podName:170864f0-3e0b-4d2e-a971-2edad85e5df2 nodeName:}" failed. No retries permitted until 2026-09-09 08:40:27.847052103 +0000 UTC m=+71.680415645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/170864f0-3e0b-4d2e-a971-2edad85e5df2-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-nfs8l" (UID: "170864f0-3e0b-4d2e-a971-2edad85e5df2") : secret "monitoring-plugin-cert" not found Sep 09 08:40:27.355902 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.355879 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc5wj\" (UniqueName: \"kubernetes.io/projected/170864f0-3e0b-4d2e-a971-2edad85e5df2-kube-api-access-gc5wj\") pod \"monitoring-plugin-85bbbbbf4-nfs8l\" (UID: \"170864f0-3e0b-4d2e-a971-2edad85e5df2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:27.579930 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:27.579894 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ddd96dc_9c23_44d0_909b_62c0551bdff6.slice/crio-4d673b8d2954bb35fd4f444daa194e2505c1d0c186a4781c7d9146b68fdc89cc WatchSource:0}: Error finding container 4d673b8d2954bb35fd4f444daa194e2505c1d0c186a4781c7d9146b68fdc89cc: Status 404 returned error can't find the container with id 4d673b8d2954bb35fd4f444daa194e2505c1d0c186a4781c7d9146b68fdc89cc Sep 09 08:40:27.635430 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.634122 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9b4698bf4-sldlc"] Sep 09 08:40:27.644618 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.644279 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.647873 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.647850 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 09 08:40:27.648890 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.648460 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9b4698bf4-sldlc"] Sep 09 08:40:27.648890 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.648617 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 09 08:40:27.648890 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.648793 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 09 08:40:27.648890 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.648842 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 09 08:40:27.649170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.648891 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 09 08:40:27.649170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.649065 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-szx82\"" Sep 09 08:40:27.653205 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.653018 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 09 08:40:27.750927 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.750896 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-telemeter-client-tls\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.751406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.750949 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-metrics-client-ca\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.751406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.751036 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.751406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.751074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.751406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.751120 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-secret-telemeter-client\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.751406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.751160 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-federate-client-tls\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.751406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.751258 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-serving-certs-ca-bundle\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.751406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.751285 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26hmx\" (UniqueName: \"kubernetes.io/projected/45773a22-188c-4017-be16-31e2f3d4714b-kube-api-access-26hmx\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.852520 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-secret-telemeter-client\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.852686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852531 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-federate-client-tls\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.852686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852576 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-serving-certs-ca-bundle\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.852686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26hmx\" (UniqueName: \"kubernetes.io/projected/45773a22-188c-4017-be16-31e2f3d4714b-kube-api-access-26hmx\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.852844 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-telemeter-client-tls\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.852844 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/170864f0-3e0b-4d2e-a971-2edad85e5df2-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nfs8l\" (UID: \"170864f0-3e0b-4d2e-a971-2edad85e5df2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:27.852844 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-metrics-client-ca\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.852844 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852827 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.853050 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.852857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.854584 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.854021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-serving-certs-ca-bundle\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.854584 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.854171 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-metrics-client-ca\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.861142 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.856666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-secret-telemeter-client\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.861142 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.860389 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45773a22-188c-4017-be16-31e2f3d4714b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.863339 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.863290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.863339 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.863325 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-federate-client-tls\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.864613 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.864566 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/45773a22-188c-4017-be16-31e2f3d4714b-telemeter-client-tls\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.866725 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.866676 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/170864f0-3e0b-4d2e-a971-2edad85e5df2-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nfs8l\" (UID: \"170864f0-3e0b-4d2e-a971-2edad85e5df2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:27.866725 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.866691 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26hmx\" (UniqueName: \"kubernetes.io/projected/45773a22-188c-4017-be16-31e2f3d4714b-kube-api-access-26hmx\") pod \"telemeter-client-9b4698bf4-sldlc\" (UID: \"45773a22-188c-4017-be16-31e2f3d4714b\") " pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:27.964205 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:27.964105 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" Sep 09 08:40:28.025836 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.025785 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wqxmh" event={"ID":"a4992530-e8b6-4ebd-bc09-04038870dfbf","Type":"ContainerStarted","Data":"1ceb2ea241451c32a00bad6154975007f899e9dbd2b56b00da75a13fe02686bc"} Sep 09 08:40:28.027480 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.027449 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerStarted","Data":"4d673b8d2954bb35fd4f444daa194e2505c1d0c186a4781c7d9146b68fdc89cc"} Sep 09 08:40:28.107489 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.107457 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:28.642712 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.642673 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:40:28.647456 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.647431 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.650176 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.650153 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 08:40:28.650351 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.650302 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 08:40:28.650425 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.650354 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-703h3cils70l3\"" Sep 09 08:40:28.650492 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.650450 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 08:40:28.650557 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.650162 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 08:40:28.651016 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.650721 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zxmk7\"" Sep 09 08:40:28.651016 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.650741 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 08:40:28.651016 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.650863 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 08:40:28.651657 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.651569 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 08:40:28.653441 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.651932 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 08:40:28.653441 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.652165 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 08:40:28.653441 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.652172 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 08:40:28.653441 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.652333 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 08:40:28.653441 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.652582 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 08:40:28.653441 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.653249 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 08:40:28.655043 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.655025 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 08:40:28.664104 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.664078 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:40:28.762184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762145 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762216 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762278 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762320 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762362 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqb2g\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-kube-api-access-dqb2g\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-web-config\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762638 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.762677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762673 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.763170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762716 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.763170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.763170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762776 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.763170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.763170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762876 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.763170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762903 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config-out\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.763170 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.762925 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.863561 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863522 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.863734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-web-config\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.863734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.863734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863626 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.863734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.863734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863682 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.863734 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863713 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863745 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863778 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config-out\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.863995 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864062 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.864024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864431 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.864073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864431 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.864135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864431 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.864177 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.864431 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.864200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqb2g\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-kube-api-access-dqb2g\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.865449 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.865422 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.865931 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.865907 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.867224 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.866546 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.867224 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.867023 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.868085 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.867740 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.868554 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.868525 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.870686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.870663 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.871928 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.871901 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-web-config\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.872372 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.872344 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.872765 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.872721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.872857 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.872795 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config-out\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.873419 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.873399 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.874019 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.874000 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.874566 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.874539 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.874644 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.874624 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.875413 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.875393 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.876043 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.876017 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.876256 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.876234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqb2g\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-kube-api-access-dqb2g\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.877631 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.877591 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.878196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.878175 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:28.960725 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:28.960635 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:29.207441 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:29.207409 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7027bc_de24_49d0_bc2a_ff7bba8f75b5.slice/crio-b2c5be7b3cf966b5a3c35dd5af9977e13b93b0f25193f331c9d48e0474028142 WatchSource:0}: Error finding container b2c5be7b3cf966b5a3c35dd5af9977e13b93b0f25193f331c9d48e0474028142: Status 404 returned error can't find the container with id b2c5be7b3cf966b5a3c35dd5af9977e13b93b0f25193f331c9d48e0474028142 Sep 09 08:40:30.037359 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:30.037315 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" event={"ID":"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5","Type":"ContainerStarted","Data":"b2c5be7b3cf966b5a3c35dd5af9977e13b93b0f25193f331c9d48e0474028142"} Sep 09 08:40:37.081423 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:37.081316 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" event={"ID":"740bc940-3f52-4ad9-8a98-ad998acdd9a0","Type":"ContainerStarted","Data":"6a644ccd65d1ecf4d288808b84cd724917297a1c6654bfc329fa2aef7cde0f20"} Sep 09 08:40:37.082555 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:37.082250 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z"] Sep 09 08:40:37.094145 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:37.093233 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7ffb9bcb-9cl22"] Sep 09 08:40:37.095614 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:37.095580 2577 generic.go:356] "Generic (PLEG): container finished" podID="c61504b7-2791-49ff-9631-c5ef7b12c6a8" containerID="ef2f2048a170f14587f9034ecabf6b7d4cde1478bbffaca52aa259c567296f84" exitCode=0 Sep 09 08:40:37.095715 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:37.095637 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kp2sc" event={"ID":"c61504b7-2791-49ff-9631-c5ef7b12c6a8","Type":"ContainerDied","Data":"ef2f2048a170f14587f9034ecabf6b7d4cde1478bbffaca52aa259c567296f84"} Sep 09 08:40:37.179935 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:37.179636 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l"] Sep 09 08:40:37.232542 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:37.232171 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:40:37.240724 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:37.240674 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8caae4fa_18d8_4b09_88a8_e787efe60fbe.slice/crio-e06ffcae7498a9de9d1373629f12a607e9d44c8eae4515ca9ecb75437ef75606 WatchSource:0}: Error finding container e06ffcae7498a9de9d1373629f12a607e9d44c8eae4515ca9ecb75437ef75606: Status 404 returned error can't find the container with id e06ffcae7498a9de9d1373629f12a607e9d44c8eae4515ca9ecb75437ef75606 Sep 09 08:40:37.248365 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:37.248335 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9b4698bf4-sldlc"] Sep 09 08:40:37.253545 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:37.253499 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45773a22_188c_4017_be16_31e2f3d4714b.slice/crio-7e7f4dc7838d3949631dec3e9c264ace6f53db8da659eb7fb3a065a93a1cade1 WatchSource:0}: Error finding container 7e7f4dc7838d3949631dec3e9c264ace6f53db8da659eb7fb3a065a93a1cade1: Status 404 returned error can't find the container with id 7e7f4dc7838d3949631dec3e9c264ace6f53db8da659eb7fb3a065a93a1cade1 Sep 09 08:40:38.108490 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.107739 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" event={"ID":"740bc940-3f52-4ad9-8a98-ad998acdd9a0","Type":"ContainerStarted","Data":"81f3041226368993bf15b62f474234eb1273f090da16182ab6a2820f12f7847c"} Sep 09 08:40:38.111195 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.110614 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9w6g4" event={"ID":"dbeeae9e-b538-462d-a818-c8914cfdc325","Type":"ContainerStarted","Data":"49185b4a3e2ba26c836a717556a41fc015aec94d38bc4deb6e46db15171117db"} Sep 09 08:40:38.111195 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.111142 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:40:38.116272 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.116208 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" event={"ID":"170864f0-3e0b-4d2e-a971-2edad85e5df2","Type":"ContainerStarted","Data":"59ff005922966217773d8a1d5a434562bc766438bdea420f07c1114c08d7f001"} Sep 09 08:40:38.118097 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.118029 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" event={"ID":"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc","Type":"ContainerStarted","Data":"41698d55cacefd8df040a2f3f39932fb27aaf8ec2c3a870864d8657f4764d4ce"} Sep 09 08:40:38.120651 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.119765 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerID="ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333" exitCode=0 Sep 09 08:40:38.120651 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.119836 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerDied","Data":"ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333"} Sep 09 08:40:38.132309 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.131558 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfdk4" podStartSLOduration=13.737809458 podStartE2EDuration="16.131538744s" totalImagesPullingTime="2.393729286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:38.128503476 +0000 UTC m=+81.961867059" watchObservedRunningTime="2026-09-09 08:40:38.131538744 +0000 UTC m=+81.964902305" Sep 09 08:40:38.152379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.152298 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7ffb9bcb-9cl22" event={"ID":"3f738416-635d-4718-bfda-5eb1c058aa8f","Type":"ContainerStarted","Data":"593f6321b25baf6495d13ecf9f6d39d3159ba7581fa93ceed1a62e85d02d4340"} Sep 09 08:40:38.173146 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.173112 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-4rmzn" event={"ID":"743d622f-3fd6-4d52-be02-fe88084a7cc2","Type":"ContainerStarted","Data":"b20d58fa9a676db7d8322aada768f13baa5f94a14324532c16eb73ffc0226bbb"} Sep 09 08:40:38.177131 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.176856 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-4rmzn" Sep 09 08:40:38.183600 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.183552 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9w6g4" podStartSLOduration=67.743691694 podStartE2EDuration="1m21.183534529s" totalImagesPullingTime="13.439842835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:38.182399093 +0000 UTC m=+82.015762667" watchObservedRunningTime="2026-09-09 08:40:38.183534529 +0000 UTC m=+82.016898089" Sep 09 08:40:38.189770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.189747 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-4rmzn" Sep 09 08:40:38.192887 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.192862 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c25ds" event={"ID":"6e9b855f-9ef8-4c2f-903d-492a4119d176","Type":"ContainerStarted","Data":"17c0237ea982eaa0abf491ecdfbd876a9a86a3b33639202b8d9a5103b1fcfdcf"} Sep 09 08:40:38.199018 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.198443 2577 generic.go:356] "Generic (PLEG): container finished" podID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" exitCode=0 Sep 09 08:40:38.199018 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.198526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerDied","Data":"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132"} Sep 09 08:40:38.199018 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.198585 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerStarted","Data":"e06ffcae7498a9de9d1373629f12a607e9d44c8eae4515ca9ecb75437ef75606"} Sep 09 08:40:38.203557 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.201591 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-4rmzn" podStartSLOduration=2.357562063 podStartE2EDuration="22.201571426s" totalImagesPullingTime="19.844009363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:38.200001177 +0000 UTC m=+82.033364754" watchObservedRunningTime="2026-09-09 08:40:38.201571426 +0000 UTC m=+82.034934985" Sep 09 08:40:38.208015 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.207922 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wqxmh" event={"ID":"a4992530-e8b6-4ebd-bc09-04038870dfbf","Type":"ContainerStarted","Data":"2baf3d6004a7266145c516a926d85b72d561b6fc884a0cc106d0b26937ecc65e"} Sep 09 08:40:38.215103 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.215061 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c25ds" podStartSLOduration=67.710383423 podStartE2EDuration="1m21.21504367s" totalImagesPullingTime="13.504660247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:38.21418477 +0000 UTC m=+82.047548356" watchObservedRunningTime="2026-09-09 08:40:38.21504367 +0000 UTC m=+82.048407213" Sep 09 08:40:38.216722 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.216640 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kp2sc" event={"ID":"c61504b7-2791-49ff-9631-c5ef7b12c6a8","Type":"ContainerStarted","Data":"f8c0f249ed6999b5827da2ede704a0daf2bcbeb4de6015bb4717b943c9882fc8"} Sep 09 08:40:38.216722 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.216692 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kp2sc" event={"ID":"c61504b7-2791-49ff-9631-c5ef7b12c6a8","Type":"ContainerStarted","Data":"c99874d450b3b8492b4c7de6d32512b466c74e8d83be905e5e89cea693311db7"} Sep 09 08:40:38.219462 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.219425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" event={"ID":"45773a22-188c-4017-be16-31e2f3d4714b","Type":"ContainerStarted","Data":"7e7f4dc7838d3949631dec3e9c264ace6f53db8da659eb7fb3a065a93a1cade1"} Sep 09 08:40:38.305985 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.305063 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kp2sc" podStartSLOduration=13.552535757 podStartE2EDuration="16.305039892s" totalImagesPullingTime="2.681348222s" totalInitContainerRuntime="71.155913ms" isStatefulPod=true podCreationTimestamp="2026-09-09 08:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:38.30321089 +0000 UTC m=+82.136574464" watchObservedRunningTime="2026-09-09 08:40:38.305039892 +0000 UTC m=+82.138403453" Sep 09 08:40:38.305985 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:38.305793 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wqxmh" podStartSLOduration=78.584292253 podStartE2EDuration="1m21.305782211s" totalImagesPullingTime="2.721489958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:38.280891491 +0000 UTC m=+82.114255051" watchObservedRunningTime="2026-09-09 08:40:38.305782211 +0000 UTC m=+82.139145771" Sep 09 08:40:45.266686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:45.266603 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" event={"ID":"45773a22-188c-4017-be16-31e2f3d4714b","Type":"ContainerStarted","Data":"6f41b529d26092b53e337c671fe26a38f8196f724fd6175e8da2900713c9d83a"} Sep 09 08:40:45.269506 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:45.269106 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" event={"ID":"170864f0-3e0b-4d2e-a971-2edad85e5df2","Type":"ContainerStarted","Data":"c671dc93fba43f558f1801722f7a6ab77173b8a21defdcf77c3baa15a26718ad"} Sep 09 08:40:45.269506 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:45.269463 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:45.274366 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:45.274254 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" event={"ID":"0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc","Type":"ContainerStarted","Data":"d38bfcc99f8502b411f9649d4b606dcaf90f2f7b2035fa66d0e3bdda4bb78ca1"} Sep 09 08:40:45.276637 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:45.276499 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" Sep 09 08:40:45.277485 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:45.277409 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" event={"ID":"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5","Type":"ContainerStarted","Data":"ac68364925b59a2daa44d84539b44012de2f6f495d5ab2ccfbddec1eb625d99f"} Sep 09 08:40:45.304088 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:45.304027 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nfs8l" podStartSLOduration=11.135136403 podStartE2EDuration="18.303996743s" totalImagesPullingTime="7.16886034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:45.286211665 +0000 UTC m=+89.119575249" watchObservedRunningTime="2026-09-09 08:40:45.303996743 +0000 UTC m=+89.137360309" Sep 09 08:40:45.305229 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:45.304352 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" podStartSLOduration=12.052529399 podStartE2EDuration="19.304342486s" totalImagesPullingTime="7.251813087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:45.302567244 +0000 UTC m=+89.135930826" watchObservedRunningTime="2026-09-09 08:40:45.304342486 +0000 UTC m=+89.137706046" Sep 09 08:40:47.156084 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.156041 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:47.156361 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.156103 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:40:47.291945 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.291905 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" event={"ID":"45773a22-188c-4017-be16-31e2f3d4714b","Type":"ContainerStarted","Data":"38be4273031a953ca53b5c7c7b7d8ba04f5f62240de4251150bebee6985690c1"} Sep 09 08:40:47.292117 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.291997 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" event={"ID":"45773a22-188c-4017-be16-31e2f3d4714b","Type":"ContainerStarted","Data":"32782887a89a642330edc4299174c2fed50ec11df17110247d07bf0890442362"} Sep 09 08:40:47.295604 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.295580 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerStarted","Data":"d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac"} Sep 09 08:40:47.295711 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.295647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerStarted","Data":"b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b"} Sep 09 08:40:47.297968 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.297937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7ffb9bcb-9cl22" event={"ID":"3f738416-635d-4718-bfda-5eb1c058aa8f","Type":"ContainerStarted","Data":"00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f"} Sep 09 08:40:47.300276 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.300256 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" event={"ID":"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5","Type":"ContainerStarted","Data":"cf5ef6dea7c76a3cc19e917c06b2bb7c9d66f2b4149c2648e869a5ef4b42a570"} Sep 09 08:40:47.300464 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.300443 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" event={"ID":"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5","Type":"ContainerStarted","Data":"142915672b9e132d57c0de94f05ca24ecbd6edb3beefb76d97ab86ea7cb59b62"} Sep 09 08:40:47.302698 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.302682 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerStarted","Data":"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e"} Sep 09 08:40:47.302813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.302802 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerStarted","Data":"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7"} Sep 09 08:40:47.315220 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.315178 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9b4698bf4-sldlc" podStartSLOduration=13.208423267 podStartE2EDuration="20.315163937s" totalImagesPullingTime="7.10674067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:47.313721289 +0000 UTC m=+91.147084860" watchObservedRunningTime="2026-09-09 08:40:47.315163937 +0000 UTC m=+91.148527487" Sep 09 08:40:47.332399 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:47.332255 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b7ffb9bcb-9cl22" podStartSLOduration=11.595252466 podStartE2EDuration="21.332241281s" totalImagesPullingTime="9.736988815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:47.329847587 +0000 UTC m=+91.163211171" watchObservedRunningTime="2026-09-09 08:40:47.332241281 +0000 UTC m=+91.165604840" Sep 09 08:40:48.310385 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.310346 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerStarted","Data":"84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989"} Sep 09 08:40:48.310781 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.310456 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerStarted","Data":"74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175"} Sep 09 08:40:48.310781 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.310491 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerStarted","Data":"5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8"} Sep 09 08:40:48.313525 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.313499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerStarted","Data":"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d"} Sep 09 08:40:48.313630 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.313556 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerStarted","Data":"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17"} Sep 09 08:40:48.313630 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.313583 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerStarted","Data":"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951"} Sep 09 08:40:48.313630 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.313607 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerStarted","Data":"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d"} Sep 09 08:40:48.341770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.341727 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=10.982913583 podStartE2EDuration="20.341712004s" totalImagesPullingTime="9.327061933s" totalInitContainerRuntime="31.736488ms" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:48.338519583 +0000 UTC m=+92.171883187" watchObservedRunningTime="2026-09-09 08:40:48.341712004 +0000 UTC m=+92.175075563" Sep 09 08:40:48.961530 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:48.961450 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:40:49.079391 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:49.079358 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7ffb9bcb-9cl22"] Sep 09 08:40:49.321544 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:49.321504 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerStarted","Data":"191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92"} Sep 09 08:40:49.324581 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:49.324547 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" event={"ID":"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5","Type":"ContainerStarted","Data":"e6e821c8837feaa38cde7d5d38cb3f3a89f2b131ec825f65edae26a3ba196089"} Sep 09 08:40:49.324726 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:49.324625 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" event={"ID":"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5","Type":"ContainerStarted","Data":"c6780015d305c581aeb1ce83456567cf598d8e60a360889f1ec0ef36dbc5d94f"} Sep 09 08:40:49.324726 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:49.324667 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" event={"ID":"dc7027bc-de24-49d0-bc2a-ff7bba8f75b5","Type":"ContainerStarted","Data":"c39103db5aa25f439b4e1573807fed0ff0dfc4f960c057caca4fa15248ab4456"} Sep 09 08:40:49.348312 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:49.348255 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.396714586 podStartE2EDuration="26.348233451s" totalImagesPullingTime="19.937870485s" totalInitContainerRuntime="13.64838ms" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:49.345282315 +0000 UTC m=+93.178645873" watchObservedRunningTime="2026-09-09 08:40:49.348233451 +0000 UTC m=+93.181597010" Sep 09 08:40:49.368348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:49.368275 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" podStartSLOduration=7.857319871 podStartE2EDuration="24.368257175s" totalImagesPullingTime="16.510937304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:49.366853933 +0000 UTC m=+93.200217538" watchObservedRunningTime="2026-09-09 08:40:49.368257175 +0000 UTC m=+93.201620734" Sep 09 08:40:50.329521 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:50.329487 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:51.338243 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:51.338216 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-76bcd6d8d4-kbr4s" Sep 09 08:40:53.286152 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.286119 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c4cb6d6d-c44dz"] Sep 09 08:40:53.320864 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.320826 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4cb6d6d-c44dz"] Sep 09 08:40:53.321049 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.320925 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.330634 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.330605 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 09 08:40:53.436648 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.436604 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-service-ca\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.436821 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.436681 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-serving-cert\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.436821 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.436717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-oauth-config\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.436821 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.436799 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t6zg\" (UniqueName: \"kubernetes.io/projected/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-kube-api-access-5t6zg\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.437005 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.436825 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-config\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.437005 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.436855 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-trusted-ca-bundle\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.437005 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.436882 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-oauth-serving-cert\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.537862 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.537772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t6zg\" (UniqueName: \"kubernetes.io/projected/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-kube-api-access-5t6zg\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.537862 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.537814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-config\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.537862 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.537844 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-trusted-ca-bundle\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.538174 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.537873 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-oauth-serving-cert\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.538174 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.537942 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-service-ca\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.538174 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.537991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-serving-cert\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.538174 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.538021 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-oauth-config\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.538801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.538759 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-config\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.538916 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.538760 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-oauth-serving-cert\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.538997 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.538951 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-service-ca\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.539610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.539584 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-trusted-ca-bundle\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.542257 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.542094 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-oauth-config\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.542377 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.542313 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-serving-cert\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.546210 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.546187 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t6zg\" (UniqueName: \"kubernetes.io/projected/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-kube-api-access-5t6zg\") pod \"console-6c4cb6d6d-c44dz\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.635718 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.635679 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:40:53.939317 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:53.939257 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4cb6d6d-c44dz"] Sep 09 08:40:53.942302 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:40:53.942260 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82ce7cc_1e71_42e7_9a66_30839d1f11c0.slice/crio-1832797f3f1b491ca3fadeb529262497d938cab440745ddf70cc27bb72e57c93 WatchSource:0}: Error finding container 1832797f3f1b491ca3fadeb529262497d938cab440745ddf70cc27bb72e57c93: Status 404 returned error can't find the container with id 1832797f3f1b491ca3fadeb529262497d938cab440745ddf70cc27bb72e57c93 Sep 09 08:40:54.346881 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:54.346837 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4cb6d6d-c44dz" event={"ID":"e82ce7cc-1e71-42e7-9a66-30839d1f11c0","Type":"ContainerStarted","Data":"ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9"} Sep 09 08:40:54.347253 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:54.346896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4cb6d6d-c44dz" event={"ID":"e82ce7cc-1e71-42e7-9a66-30839d1f11c0","Type":"ContainerStarted","Data":"1832797f3f1b491ca3fadeb529262497d938cab440745ddf70cc27bb72e57c93"} Sep 09 08:40:54.364961 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:54.364915 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c4cb6d6d-c44dz" podStartSLOduration=1.364900793 podStartE2EDuration="1.364900793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:40:54.363767379 +0000 UTC m=+98.197130968" watchObservedRunningTime="2026-09-09 08:40:54.364900793 +0000 UTC m=+98.198264351" Sep 09 08:40:57.052504 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:40:57.052462 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:41:03.636426 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:03.636382 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:41:03.636948 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:03.636452 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:41:03.641403 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:03.641384 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:41:04.394930 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:04.394901 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:41:07.161135 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:07.161104 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:41:07.164905 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:07.164877 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7dcb6bdff4-5gj4z" Sep 09 08:41:10.239510 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:10.239479 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9w6g4" Sep 09 08:41:14.343663 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.343612 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b7ffb9bcb-9cl22" podUID="3f738416-635d-4718-bfda-5eb1c058aa8f" containerName="console" containerID="cri-o://00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f" gracePeriod=15 Sep 09 08:41:14.629731 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.629702 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:41:14.727178 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727140 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26nkt\" (UniqueName: \"kubernetes.io/projected/3f738416-635d-4718-bfda-5eb1c058aa8f-kube-api-access-26nkt\") pod \"3f738416-635d-4718-bfda-5eb1c058aa8f\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " Sep 09 08:41:14.727348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727202 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-console-config\") pod \"3f738416-635d-4718-bfda-5eb1c058aa8f\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " Sep 09 08:41:14.727348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727231 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-service-ca\") pod \"3f738416-635d-4718-bfda-5eb1c058aa8f\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " Sep 09 08:41:14.727348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727254 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-oauth-config\") pod \"3f738416-635d-4718-bfda-5eb1c058aa8f\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " Sep 09 08:41:14.727348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727299 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-oauth-serving-cert\") pod \"3f738416-635d-4718-bfda-5eb1c058aa8f\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " Sep 09 08:41:14.727348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727341 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-serving-cert\") pod \"3f738416-635d-4718-bfda-5eb1c058aa8f\" (UID: \"3f738416-635d-4718-bfda-5eb1c058aa8f\") " Sep 09 08:41:14.727646 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727615 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-service-ca" (OuterVolumeSpecName: "service-ca") pod "3f738416-635d-4718-bfda-5eb1c058aa8f" (UID: "3f738416-635d-4718-bfda-5eb1c058aa8f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:14.727757 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727659 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-console-config" (OuterVolumeSpecName: "console-config") pod "3f738416-635d-4718-bfda-5eb1c058aa8f" (UID: "3f738416-635d-4718-bfda-5eb1c058aa8f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:14.727757 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.727706 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3f738416-635d-4718-bfda-5eb1c058aa8f" (UID: "3f738416-635d-4718-bfda-5eb1c058aa8f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:14.729558 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.729527 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3f738416-635d-4718-bfda-5eb1c058aa8f" (UID: "3f738416-635d-4718-bfda-5eb1c058aa8f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:14.729650 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.729564 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f738416-635d-4718-bfda-5eb1c058aa8f-kube-api-access-26nkt" (OuterVolumeSpecName: "kube-api-access-26nkt") pod "3f738416-635d-4718-bfda-5eb1c058aa8f" (UID: "3f738416-635d-4718-bfda-5eb1c058aa8f"). InnerVolumeSpecName "kube-api-access-26nkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:41:14.729650 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.729627 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3f738416-635d-4718-bfda-5eb1c058aa8f" (UID: "3f738416-635d-4718-bfda-5eb1c058aa8f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:14.828872 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.828829 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26nkt\" (UniqueName: \"kubernetes.io/projected/3f738416-635d-4718-bfda-5eb1c058aa8f-kube-api-access-26nkt\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:14.828872 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.828866 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-console-config\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:14.828872 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.828876 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-service-ca\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:14.828872 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.828884 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-oauth-config\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:14.829155 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.828893 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f738416-635d-4718-bfda-5eb1c058aa8f-oauth-serving-cert\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:14.829155 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:14.828901 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f738416-635d-4718-bfda-5eb1c058aa8f-console-serving-cert\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:15.439234 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.439201 2577 generic.go:356] "Generic (PLEG): container finished" podID="3f738416-635d-4718-bfda-5eb1c058aa8f" containerID="00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f" exitCode=0 Sep 09 08:41:15.439640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.439259 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7ffb9bcb-9cl22" event={"ID":"3f738416-635d-4718-bfda-5eb1c058aa8f","Type":"ContainerDied","Data":"00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f"} Sep 09 08:41:15.439640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.439296 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7ffb9bcb-9cl22" event={"ID":"3f738416-635d-4718-bfda-5eb1c058aa8f","Type":"ContainerDied","Data":"593f6321b25baf6495d13ecf9f6d39d3159ba7581fa93ceed1a62e85d02d4340"} Sep 09 08:41:15.439640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.439315 2577 scope.go:117] "RemoveContainer" containerID="00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f" Sep 09 08:41:15.439640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.439261 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7ffb9bcb-9cl22" Sep 09 08:41:15.448223 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.448194 2577 scope.go:117] "RemoveContainer" containerID="00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f" Sep 09 08:41:15.448655 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:15.448623 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f\": container with ID starting with 00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f not found: ID does not exist" containerID="00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f" Sep 09 08:41:15.448753 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.448666 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f"} err="failed to get container status \"00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f\": rpc error: code = NotFound desc = could not find container \"00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f\": container with ID starting with 00f685a3c7250f43d29b79c5312803fbd04e3668f0f99b676c86d96310f7405f not found: ID does not exist" Sep 09 08:41:15.471147 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.471120 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7ffb9bcb-9cl22"] Sep 09 08:41:15.474903 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:15.474884 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b7ffb9bcb-9cl22"] Sep 09 08:41:16.633637 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:16.633605 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f738416-635d-4718-bfda-5eb1c058aa8f" path="/var/lib/kubelet/pods/3f738416-635d-4718-bfda-5eb1c058aa8f/volumes" Sep 09 08:41:28.961648 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:28.961610 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:29.037587 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:29.037559 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:29.510919 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:29.510889 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:42.753724 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:42.753688 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:41:42.754184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:42.753961 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="alertmanager" containerID="cri-o://b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b" gracePeriod=120 Sep 09 08:41:42.754184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:42.754037 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy-metric" containerID="cri-o://84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989" gracePeriod=120 Sep 09 08:41:42.754184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:42.754059 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy-web" containerID="cri-o://5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8" gracePeriod=120 Sep 09 08:41:42.754184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:42.754064 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy" containerID="cri-o://74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175" gracePeriod=120 Sep 09 08:41:42.754184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:42.754086 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="config-reloader" containerID="cri-o://d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac" gracePeriod=120 Sep 09 08:41:42.754184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:42.754181 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="prom-label-proxy" containerID="cri-o://191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92" gracePeriod=120 Sep 09 08:41:43.567259 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567227 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerID="191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92" exitCode=0 Sep 09 08:41:43.567259 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567253 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerID="84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989" exitCode=0 Sep 09 08:41:43.567259 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567259 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerID="74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175" exitCode=0 Sep 09 08:41:43.567259 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567265 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerID="d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac" exitCode=0 Sep 09 08:41:43.567507 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567271 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerID="b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b" exitCode=0 Sep 09 08:41:43.567507 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567303 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerDied","Data":"191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92"} Sep 09 08:41:43.567507 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567389 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerDied","Data":"84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989"} Sep 09 08:41:43.567507 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerDied","Data":"74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175"} Sep 09 08:41:43.567507 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567441 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerDied","Data":"d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac"} Sep 09 08:41:43.567507 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:43.567462 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerDied","Data":"b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b"} Sep 09 08:41:44.001289 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.001268 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.093934 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.093862 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-volume\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.093934 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.093897 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-main-db\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094142 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.093944 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-web\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094142 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.093996 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-web-config\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094142 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094047 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdm88\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-kube-api-access-wdm88\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094142 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094071 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-cluster-tls-config\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094142 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094100 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094393 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094231 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-metrics-client-ca\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094393 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094273 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:41:44.094393 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094283 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094393 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094348 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-tls-assets\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094392 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-out\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094426 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094482 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle\") pod \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\" (UID: \"9ddd96dc-9c23-44d0-909b-62c0551bdff6\") " Sep 09 08:41:44.094936 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094860 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-main-db\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.095072 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.094947 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:44.095727 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.095691 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:44.097196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.097167 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:44.098028 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.097986 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:44.098178 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.098041 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-kube-api-access-wdm88" (OuterVolumeSpecName: "kube-api-access-wdm88") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "kube-api-access-wdm88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:41:44.098178 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.098069 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:44.098301 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.098244 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-volume" (OuterVolumeSpecName: "config-volume") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:44.098355 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.098314 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:44.098805 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.098787 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-out" (OuterVolumeSpecName: "config-out") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:41:44.099406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.099372 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:41:44.102853 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.102826 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:44.109699 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.109673 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-web-config" (OuterVolumeSpecName: "web-config") pod "9ddd96dc-9c23-44d0-909b-62c0551bdff6" (UID: "9ddd96dc-9c23-44d0-909b-62c0551bdff6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:44.195427 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195388 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-main-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195427 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195418 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-metrics-client-ca\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195427 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195429 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195427 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195439 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-tls-assets\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195448 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-out\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195456 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195465 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ddd96dc-9c23-44d0-909b-62c0551bdff6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195473 2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-config-volume\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195482 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195491 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-web-config\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195499 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdm88\" (UniqueName: \"kubernetes.io/projected/9ddd96dc-9c23-44d0-909b-62c0551bdff6-kube-api-access-wdm88\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.195689 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.195508 2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ddd96dc-9c23-44d0-909b-62c0551bdff6-cluster-tls-config\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:44.575123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.575088 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerID="5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8" exitCode=0 Sep 09 08:41:44.575325 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.575168 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.575325 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.575166 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerDied","Data":"5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8"} Sep 09 08:41:44.575325 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.575238 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ddd96dc-9c23-44d0-909b-62c0551bdff6","Type":"ContainerDied","Data":"4d673b8d2954bb35fd4f444daa194e2505c1d0c186a4781c7d9146b68fdc89cc"} Sep 09 08:41:44.575325 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.575281 2577 scope.go:117] "RemoveContainer" containerID="191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92" Sep 09 08:41:44.583888 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.583872 2577 scope.go:117] "RemoveContainer" containerID="84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989" Sep 09 08:41:44.591016 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.590994 2577 scope.go:117] "RemoveContainer" containerID="74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175" Sep 09 08:41:44.598498 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.598456 2577 scope.go:117] "RemoveContainer" containerID="5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8" Sep 09 08:41:44.599061 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.599033 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:41:44.602809 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.602786 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:41:44.606073 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.606056 2577 scope.go:117] "RemoveContainer" containerID="d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac" Sep 09 08:41:44.613149 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.613134 2577 scope.go:117] "RemoveContainer" containerID="b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b" Sep 09 08:41:44.620220 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.620203 2577 scope.go:117] "RemoveContainer" containerID="ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333" Sep 09 08:41:44.627474 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.627436 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:41:44.627544 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.627513 2577 scope.go:117] "RemoveContainer" containerID="191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92" Sep 09 08:41:44.627851 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:44.627828 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92\": container with ID starting with 191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92 not found: ID does not exist" containerID="191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92" Sep 09 08:41:44.627952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.627855 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy" Sep 09 08:41:44.627952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.627855 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92"} err="failed to get container status \"191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92\": rpc error: code = NotFound desc = could not find container \"191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92\": container with ID starting with 191d44b4c9834716d9528d57db4389bc533100baa9ecd7c0bddf13bb1db7ef92 not found: ID does not exist" Sep 09 08:41:44.627952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.627881 2577 scope.go:117] "RemoveContainer" containerID="84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989" Sep 09 08:41:44.627952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.627870 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy" Sep 09 08:41:44.627952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.627951 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f738416-635d-4718-bfda-5eb1c058aa8f" containerName="console" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.627963 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f738416-635d-4718-bfda-5eb1c058aa8f" containerName="console" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628009 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="config-reloader" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628019 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="config-reloader" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628045 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="alertmanager" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628054 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="alertmanager" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628066 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy-web" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628075 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy-web" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628097 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy-metric" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628106 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy-metric" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628117 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="prom-label-proxy" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628125 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="prom-label-proxy" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628157 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="init-config-reloader" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628167 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="init-config-reloader" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:44.628262 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989\": container with ID starting with 84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989 not found: ID does not exist" containerID="84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628294 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989"} err="failed to get container status \"84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989\": rpc error: code = NotFound desc = could not find container \"84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989\": container with ID starting with 84150cff5eda47fd01ff30aa169fb8fff935a5272a7507a4bbd62dd9c4b1b989 not found: ID does not exist" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628315 2577 scope.go:117] "RemoveContainer" containerID="74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628275 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f738416-635d-4718-bfda-5eb1c058aa8f" containerName="console" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628372 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="prom-label-proxy" Sep 09 08:41:44.628374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628383 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy-metric" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628394 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="config-reloader" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628403 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy-web" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628415 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="kube-rbac-proxy" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628424 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" containerName="alertmanager" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:44.628621 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175\": container with ID starting with 74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175 not found: ID does not exist" containerID="74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628674 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175"} err="failed to get container status \"74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175\": rpc error: code = NotFound desc = could not find container \"74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175\": container with ID starting with 74c93b3fe5ac9bc9417a6d84963678de2de33f28f023bd2221a12f5479d4c175 not found: ID does not exist" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.628699 2577 scope.go:117] "RemoveContainer" containerID="5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:44.629024 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8\": container with ID starting with 5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8 not found: ID does not exist" containerID="5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.629058 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8"} err="failed to get container status \"5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8\": rpc error: code = NotFound desc = could not find container \"5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8\": container with ID starting with 5e2c30a50570d88d21c1be382dde07da95acbfa28d1121ae5052896e519061d8 not found: ID does not exist" Sep 09 08:41:44.629190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.629076 2577 scope.go:117] "RemoveContainer" containerID="d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac" Sep 09 08:41:44.629807 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:44.629422 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac\": container with ID starting with d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac not found: ID does not exist" containerID="d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac" Sep 09 08:41:44.629807 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.629450 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac"} err="failed to get container status \"d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac\": rpc error: code = NotFound desc = could not find container \"d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac\": container with ID starting with d416bd912abbab3d09b159cf5c003498f3477cb7f0dc4b03628eb0c868c4b3ac not found: ID does not exist" Sep 09 08:41:44.629807 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.629470 2577 scope.go:117] "RemoveContainer" containerID="b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b" Sep 09 08:41:44.629932 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:44.629827 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b\": container with ID starting with b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b not found: ID does not exist" containerID="b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b" Sep 09 08:41:44.629932 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.629865 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b"} err="failed to get container status \"b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b\": rpc error: code = NotFound desc = could not find container \"b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b\": container with ID starting with b66baa433f653fe003f74b3b51b8aa826a0d6d09f709cf3acf50b1b610731b8b not found: ID does not exist" Sep 09 08:41:44.629932 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.629884 2577 scope.go:117] "RemoveContainer" containerID="ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333" Sep 09 08:41:44.630209 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:44.630187 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333\": container with ID starting with ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333 not found: ID does not exist" containerID="ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333" Sep 09 08:41:44.630250 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.630217 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333"} err="failed to get container status \"ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333\": rpc error: code = NotFound desc = could not find container \"ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333\": container with ID starting with ae54231c01bb8a9c773a01c9073fdb9ace40371197156cf1b144f3276f64f333 not found: ID does not exist" Sep 09 08:41:44.633045 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.633027 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.635756 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.635730 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 08:41:44.635828 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.635757 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 08:41:44.636045 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.636030 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hjqx8\"" Sep 09 08:41:44.636132 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.636065 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 08:41:44.636132 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.636084 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 08:41:44.636454 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.636432 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 08:41:44.636557 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.636481 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 08:41:44.636557 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.636532 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 08:41:44.636557 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.636546 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 08:41:44.638779 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.638756 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ddd96dc-9c23-44d0-909b-62c0551bdff6" path="/var/lib/kubelet/pods/9ddd96dc-9c23-44d0-909b-62c0551bdff6/volumes" Sep 09 08:41:44.641715 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.641696 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 08:41:44.644528 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.644508 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:41:44.699801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.699769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.699801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.699798 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700079 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.699822 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700079 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.699844 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-web-config\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700079 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.699907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700079 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.699988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700079 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.700007 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700079 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.700029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700377 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.700119 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700377 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.700192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700377 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.700223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700377 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.700249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7g72\" (UniqueName: \"kubernetes.io/projected/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-kube-api-access-q7g72\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.700377 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.700276 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-config-out\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.800652 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.800615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.800801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.800669 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.800801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.800691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.800801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.800709 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7g72\" (UniqueName: \"kubernetes.io/projected/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-kube-api-access-q7g72\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.800801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.800729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-config-out\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.800801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.800759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801127 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.800959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801127 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.801039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801127 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.801088 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-web-config\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.801159 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.801163 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.801209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.801232 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.801256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.801764 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.801743 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.803251 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.803224 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.803735 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.803709 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.804046 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.804017 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.804535 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.804202 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-config-out\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.804535 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.804265 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-web-config\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.804535 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.804432 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.804713 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.804577 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.804713 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.804618 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.804822 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.804801 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.805034 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.805019 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.809384 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.809366 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7g72\" (UniqueName: \"kubernetes.io/projected/0792c64b-7c8b-4dd8-b329-2fb6919eb43d-kube-api-access-q7g72\") pod \"alertmanager-main-0\" (UID: \"0792c64b-7c8b-4dd8-b329-2fb6919eb43d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:44.944802 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:44.944770 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 08:41:45.220692 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:45.220660 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 08:41:45.221597 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:41:45.221573 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0792c64b_7c8b_4dd8_b329_2fb6919eb43d.slice/crio-3a30a9c353197613a77d771676b2d0f6f68a4d5b1c25c08300921afd6e770912 WatchSource:0}: Error finding container 3a30a9c353197613a77d771676b2d0f6f68a4d5b1c25c08300921afd6e770912: Status 404 returned error can't find the container with id 3a30a9c353197613a77d771676b2d0f6f68a4d5b1c25c08300921afd6e770912 Sep 09 08:41:45.581988 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:45.581944 2577 generic.go:356] "Generic (PLEG): container finished" podID="0792c64b-7c8b-4dd8-b329-2fb6919eb43d" containerID="db05a1c7b18af1af82e7d48799fa4304f25ec8d4803d8fcc12e43c3282a836ed" exitCode=0 Sep 09 08:41:45.582145 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:45.582012 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0792c64b-7c8b-4dd8-b329-2fb6919eb43d","Type":"ContainerDied","Data":"db05a1c7b18af1af82e7d48799fa4304f25ec8d4803d8fcc12e43c3282a836ed"} Sep 09 08:41:45.582145 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:45.582093 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0792c64b-7c8b-4dd8-b329-2fb6919eb43d","Type":"ContainerStarted","Data":"3a30a9c353197613a77d771676b2d0f6f68a4d5b1c25c08300921afd6e770912"} Sep 09 08:41:46.590271 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.590237 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0792c64b-7c8b-4dd8-b329-2fb6919eb43d","Type":"ContainerStarted","Data":"0bff6b58e31e8e9c8909303e1ca9a30864b8be064c4dc7f3acace5d3294c531e"} Sep 09 08:41:46.590704 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.590305 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0792c64b-7c8b-4dd8-b329-2fb6919eb43d","Type":"ContainerStarted","Data":"055b7c46c0b5a28d6b460c236f77581edc5c490dc3159d87e4bf0a6ca9652321"} Sep 09 08:41:46.590704 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.590333 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0792c64b-7c8b-4dd8-b329-2fb6919eb43d","Type":"ContainerStarted","Data":"fa6f2dc5ebb33f252c1de64474a660e5c3709bcb14d1d8993bf1dbd74f53cb88"} Sep 09 08:41:46.590704 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.590356 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0792c64b-7c8b-4dd8-b329-2fb6919eb43d","Type":"ContainerStarted","Data":"366b3106edb82890cf4d01d72983bd7549db496fe6d53354bc297fe9f2cecad2"} Sep 09 08:41:46.590704 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.590387 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0792c64b-7c8b-4dd8-b329-2fb6919eb43d","Type":"ContainerStarted","Data":"d2b599a2c50483c70234d9a2aa2e50a62ceee0571e5c627fbbfe654ebbe6f094"} Sep 09 08:41:46.590704 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.590416 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0792c64b-7c8b-4dd8-b329-2fb6919eb43d","Type":"ContainerStarted","Data":"d9186b90ed101f9dce3ea65c3fcd7da3e6b902f394c9254e0ea9ae3fd93864d0"} Sep 09 08:41:46.618459 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.618408 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.601796417 podStartE2EDuration="2.618391542s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.595125ms" isStatefulPod=false podCreationTimestamp="2026-09-09 08:41:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:41:46.616446356 +0000 UTC m=+150.449809928" watchObservedRunningTime="2026-09-09 08:41:46.618391542 +0000 UTC m=+150.451755100" Sep 09 08:41:46.989176 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.989085 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:41:46.989537 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.989482 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="prometheus" containerID="cri-o://50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" gracePeriod=600 Sep 09 08:41:46.989631 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.989536 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy-web" containerID="cri-o://bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" gracePeriod=600 Sep 09 08:41:46.989631 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.989536 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="thanos-sidecar" containerID="cri-o://d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" gracePeriod=600 Sep 09 08:41:46.989631 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.989506 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy" containerID="cri-o://17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" gracePeriod=600 Sep 09 08:41:46.989792 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.989521 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" gracePeriod=600 Sep 09 08:41:46.989845 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:46.989788 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="config-reloader" containerID="cri-o://560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" gracePeriod=600 Sep 09 08:41:47.236872 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.236843 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.321950 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.321921 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-metrics-client-ca\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322162 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.321966 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqb2g\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-kube-api-access-dqb2g\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322162 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322054 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-serving-certs-ca-bundle\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322162 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322088 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322162 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322108 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-kube-rbac-proxy\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322162 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322134 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-tls-assets\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322433 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322165 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-0\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322433 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322191 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config-out\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322433 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322236 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322433 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322292 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-kubelet-serving-ca-bundle\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322433 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322340 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-tls\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322433 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322365 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-trusted-ca-bundle\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322433 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322402 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:47.322433 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322428 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-grpc-tls\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322461 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-2\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322466 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:47.322808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322488 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-1\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322520 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-db\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322542 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-web-config\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322574 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322603 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-thanos-prometheus-http-client-file\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.322808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322643 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-metrics-client-certs\") pod \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\" (UID: \"8caae4fa-18d8-4b09-88a8-e787efe60fbe\") " Sep 09 08:41:47.323216 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322929 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-metrics-client-ca\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.323216 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.322949 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.324928 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.323498 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:47.324928 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.323686 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:47.324928 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.323879 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:47.324928 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.324686 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:47.325281 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.325249 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:41:47.325521 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.325493 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.325651 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.325615 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:41:47.328156 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.328129 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-kube-api-access-dqb2g" (OuterVolumeSpecName: "kube-api-access-dqb2g") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "kube-api-access-dqb2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:41:47.328301 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.328279 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config-out" (OuterVolumeSpecName: "config-out") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:41:47.328380 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.328330 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.328849 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.328452 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.328849 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.328561 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.328849 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.328691 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.328849 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.328821 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config" (OuterVolumeSpecName: "config") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.329682 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.329648 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:41:47.330640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.330426 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.330640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.330608 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.342016 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.341992 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-web-config" (OuterVolumeSpecName: "web-config") pod "8caae4fa-18d8-4b09-88a8-e787efe60fbe" (UID: "8caae4fa-18d8-4b09-88a8-e787efe60fbe"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:41:47.423525 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423485 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423525 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423518 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423525 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423530 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423539 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423549 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-grpc-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423559 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423568 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423576 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-db\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423586 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-web-config\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423594 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423602 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423612 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-metrics-client-certs\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423620 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqb2g\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-kube-api-access-dqb2g\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423629 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423637 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8caae4fa-18d8-4b09-88a8-e787efe60fbe-secret-kube-rbac-proxy\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423645 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8caae4fa-18d8-4b09-88a8-e787efe60fbe-tls-assets\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423672 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8caae4fa-18d8-4b09-88a8-e787efe60fbe-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.423850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.423680 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8caae4fa-18d8-4b09-88a8-e787efe60fbe-config-out\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:41:47.598621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598537 2577 generic.go:356] "Generic (PLEG): container finished" podID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" exitCode=0 Sep 09 08:41:47.598621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598562 2577 generic.go:356] "Generic (PLEG): container finished" podID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" exitCode=0 Sep 09 08:41:47.598621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598569 2577 generic.go:356] "Generic (PLEG): container finished" podID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" exitCode=0 Sep 09 08:41:47.598621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598575 2577 generic.go:356] "Generic (PLEG): container finished" podID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" exitCode=0 Sep 09 08:41:47.598621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598580 2577 generic.go:356] "Generic (PLEG): container finished" podID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" exitCode=0 Sep 09 08:41:47.598621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598585 2577 generic.go:356] "Generic (PLEG): container finished" podID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" exitCode=0 Sep 09 08:41:47.598621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598611 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.598621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598614 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerDied","Data":"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d"} Sep 09 08:41:47.599398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598703 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerDied","Data":"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17"} Sep 09 08:41:47.599398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598752 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerDied","Data":"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951"} Sep 09 08:41:47.599398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598765 2577 scope.go:117] "RemoveContainer" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" Sep 09 08:41:47.599398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598793 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerDied","Data":"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d"} Sep 09 08:41:47.599398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598832 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerDied","Data":"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e"} Sep 09 08:41:47.599398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598871 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerDied","Data":"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7"} Sep 09 08:41:47.599398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.598913 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8caae4fa-18d8-4b09-88a8-e787efe60fbe","Type":"ContainerDied","Data":"e06ffcae7498a9de9d1373629f12a607e9d44c8eae4515ca9ecb75437ef75606"} Sep 09 08:41:47.607150 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.607125 2577 scope.go:117] "RemoveContainer" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" Sep 09 08:41:47.614343 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.614328 2577 scope.go:117] "RemoveContainer" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" Sep 09 08:41:47.621254 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.621226 2577 scope.go:117] "RemoveContainer" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" Sep 09 08:41:47.624395 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.624371 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:41:47.628598 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.628534 2577 scope.go:117] "RemoveContainer" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" Sep 09 08:41:47.630966 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.630943 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:41:47.635449 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.635432 2577 scope.go:117] "RemoveContainer" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" Sep 09 08:41:47.642369 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.642355 2577 scope.go:117] "RemoveContainer" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" Sep 09 08:41:47.649108 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.649093 2577 scope.go:117] "RemoveContainer" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" Sep 09 08:41:47.649362 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:47.649346 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": container with ID starting with b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d not found: ID does not exist" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" Sep 09 08:41:47.649406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.649370 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d"} err="failed to get container status \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": rpc error: code = NotFound desc = could not find container \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": container with ID starting with b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d not found: ID does not exist" Sep 09 08:41:47.649406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.649387 2577 scope.go:117] "RemoveContainer" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" Sep 09 08:41:47.649619 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:47.649605 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": container with ID starting with 17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17 not found: ID does not exist" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" Sep 09 08:41:47.649661 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.649622 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17"} err="failed to get container status \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": rpc error: code = NotFound desc = could not find container \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": container with ID starting with 17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17 not found: ID does not exist" Sep 09 08:41:47.649661 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.649634 2577 scope.go:117] "RemoveContainer" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" Sep 09 08:41:47.649868 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:47.649843 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": container with ID starting with bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951 not found: ID does not exist" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" Sep 09 08:41:47.649906 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.649872 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951"} err="failed to get container status \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": rpc error: code = NotFound desc = could not find container \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": container with ID starting with bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951 not found: ID does not exist" Sep 09 08:41:47.649906 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.649882 2577 scope.go:117] "RemoveContainer" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" Sep 09 08:41:47.650168 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:47.650153 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": container with ID starting with d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d not found: ID does not exist" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" Sep 09 08:41:47.650221 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.650171 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d"} err="failed to get container status \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": rpc error: code = NotFound desc = could not find container \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": container with ID starting with d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d not found: ID does not exist" Sep 09 08:41:47.650221 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.650193 2577 scope.go:117] "RemoveContainer" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" Sep 09 08:41:47.650431 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:47.650417 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": container with ID starting with 560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e not found: ID does not exist" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" Sep 09 08:41:47.650479 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.650433 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e"} err="failed to get container status \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": rpc error: code = NotFound desc = could not find container \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": container with ID starting with 560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e not found: ID does not exist" Sep 09 08:41:47.650479 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.650448 2577 scope.go:117] "RemoveContainer" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" Sep 09 08:41:47.650672 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:47.650658 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": container with ID starting with 50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7 not found: ID does not exist" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" Sep 09 08:41:47.650710 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.650674 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7"} err="failed to get container status \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": rpc error: code = NotFound desc = could not find container \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": container with ID starting with 50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7 not found: ID does not exist" Sep 09 08:41:47.650710 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.650684 2577 scope.go:117] "RemoveContainer" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" Sep 09 08:41:47.650904 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:41:47.650891 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": container with ID starting with d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132 not found: ID does not exist" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" Sep 09 08:41:47.650944 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.650906 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132"} err="failed to get container status \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": rpc error: code = NotFound desc = could not find container \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": container with ID starting with d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132 not found: ID does not exist" Sep 09 08:41:47.650944 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.650917 2577 scope.go:117] "RemoveContainer" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" Sep 09 08:41:47.651155 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.651137 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d"} err="failed to get container status \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": rpc error: code = NotFound desc = could not find container \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": container with ID starting with b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d not found: ID does not exist" Sep 09 08:41:47.651195 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.651154 2577 scope.go:117] "RemoveContainer" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" Sep 09 08:41:47.651388 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.651366 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17"} err="failed to get container status \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": rpc error: code = NotFound desc = could not find container \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": container with ID starting with 17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17 not found: ID does not exist" Sep 09 08:41:47.651428 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.651391 2577 scope.go:117] "RemoveContainer" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" Sep 09 08:41:47.651605 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.651589 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951"} err="failed to get container status \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": rpc error: code = NotFound desc = could not find container \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": container with ID starting with bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951 not found: ID does not exist" Sep 09 08:41:47.651647 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.651605 2577 scope.go:117] "RemoveContainer" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" Sep 09 08:41:47.651827 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.651813 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d"} err="failed to get container status \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": rpc error: code = NotFound desc = could not find container \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": container with ID starting with d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d not found: ID does not exist" Sep 09 08:41:47.651867 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.651826 2577 scope.go:117] "RemoveContainer" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" Sep 09 08:41:47.652042 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652025 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e"} err="failed to get container status \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": rpc error: code = NotFound desc = could not find container \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": container with ID starting with 560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e not found: ID does not exist" Sep 09 08:41:47.652102 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652043 2577 scope.go:117] "RemoveContainer" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" Sep 09 08:41:47.652265 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652249 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7"} err="failed to get container status \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": rpc error: code = NotFound desc = could not find container \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": container with ID starting with 50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7 not found: ID does not exist" Sep 09 08:41:47.652312 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652265 2577 scope.go:117] "RemoveContainer" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" Sep 09 08:41:47.652450 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652432 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132"} err="failed to get container status \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": rpc error: code = NotFound desc = could not find container \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": container with ID starting with d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132 not found: ID does not exist" Sep 09 08:41:47.652491 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652450 2577 scope.go:117] "RemoveContainer" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" Sep 09 08:41:47.652634 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652619 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d"} err="failed to get container status \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": rpc error: code = NotFound desc = could not find container \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": container with ID starting with b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d not found: ID does not exist" Sep 09 08:41:47.652678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652634 2577 scope.go:117] "RemoveContainer" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" Sep 09 08:41:47.652837 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652820 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17"} err="failed to get container status \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": rpc error: code = NotFound desc = could not find container \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": container with ID starting with 17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17 not found: ID does not exist" Sep 09 08:41:47.652883 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.652836 2577 scope.go:117] "RemoveContainer" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" Sep 09 08:41:47.653297 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.653168 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951"} err="failed to get container status \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": rpc error: code = NotFound desc = could not find container \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": container with ID starting with bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951 not found: ID does not exist" Sep 09 08:41:47.653419 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.653301 2577 scope.go:117] "RemoveContainer" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" Sep 09 08:41:47.653705 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.653678 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d"} err="failed to get container status \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": rpc error: code = NotFound desc = could not find container \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": container with ID starting with d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d not found: ID does not exist" Sep 09 08:41:47.653705 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.653705 2577 scope.go:117] "RemoveContainer" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" Sep 09 08:41:47.654070 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.654037 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e"} err="failed to get container status \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": rpc error: code = NotFound desc = could not find container \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": container with ID starting with 560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e not found: ID does not exist" Sep 09 08:41:47.654070 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.654061 2577 scope.go:117] "RemoveContainer" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" Sep 09 08:41:47.654313 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.654296 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7"} err="failed to get container status \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": rpc error: code = NotFound desc = could not find container \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": container with ID starting with 50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7 not found: ID does not exist" Sep 09 08:41:47.654313 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.654312 2577 scope.go:117] "RemoveContainer" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" Sep 09 08:41:47.654558 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.654540 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132"} err="failed to get container status \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": rpc error: code = NotFound desc = could not find container \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": container with ID starting with d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132 not found: ID does not exist" Sep 09 08:41:47.654604 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.654559 2577 scope.go:117] "RemoveContainer" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" Sep 09 08:41:47.654809 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.654790 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d"} err="failed to get container status \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": rpc error: code = NotFound desc = could not find container \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": container with ID starting with b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d not found: ID does not exist" Sep 09 08:41:47.654854 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.654810 2577 scope.go:117] "RemoveContainer" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" Sep 09 08:41:47.655111 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655089 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17"} err="failed to get container status \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": rpc error: code = NotFound desc = could not find container \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": container with ID starting with 17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17 not found: ID does not exist" Sep 09 08:41:47.655111 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655110 2577 scope.go:117] "RemoveContainer" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" Sep 09 08:41:47.655325 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655307 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951"} err="failed to get container status \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": rpc error: code = NotFound desc = could not find container \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": container with ID starting with bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951 not found: ID does not exist" Sep 09 08:41:47.655395 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655326 2577 scope.go:117] "RemoveContainer" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" Sep 09 08:41:47.655446 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655393 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:41:47.655579 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655563 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d"} err="failed to get container status \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": rpc error: code = NotFound desc = could not find container \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": container with ID starting with d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d not found: ID does not exist" Sep 09 08:41:47.655621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655580 2577 scope.go:117] "RemoveContainer" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" Sep 09 08:41:47.655768 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655750 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy" Sep 09 08:41:47.655831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655771 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy" Sep 09 08:41:47.655831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655784 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy-thanos" Sep 09 08:41:47.655831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655789 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy-thanos" Sep 09 08:41:47.655831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655800 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy-web" Sep 09 08:41:47.655831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655805 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy-web" Sep 09 08:41:47.655831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655815 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="config-reloader" Sep 09 08:41:47.655831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655811 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e"} err="failed to get container status \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": rpc error: code = NotFound desc = could not find container \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": container with ID starting with 560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e not found: ID does not exist" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655833 2577 scope.go:117] "RemoveContainer" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655822 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="config-reloader" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655887 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="prometheus" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655895 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="prometheus" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655909 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="init-config-reloader" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655915 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="init-config-reloader" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655921 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="thanos-sidecar" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.655926 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="thanos-sidecar" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656028 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="thanos-sidecar" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656041 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="prometheus" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656052 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="config-reloader" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656057 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656069 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy-web" Sep 09 08:41:47.656123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656075 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" containerName="kube-rbac-proxy-thanos" Sep 09 08:41:47.656733 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656129 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7"} err="failed to get container status \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": rpc error: code = NotFound desc = could not find container \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": container with ID starting with 50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7 not found: ID does not exist" Sep 09 08:41:47.656733 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656155 2577 scope.go:117] "RemoveContainer" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" Sep 09 08:41:47.656733 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656429 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132"} err="failed to get container status \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": rpc error: code = NotFound desc = could not find container \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": container with ID starting with d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132 not found: ID does not exist" Sep 09 08:41:47.656733 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656459 2577 scope.go:117] "RemoveContainer" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" Sep 09 08:41:47.656733 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656719 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d"} err="failed to get container status \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": rpc error: code = NotFound desc = could not find container \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": container with ID starting with b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d not found: ID does not exist" Sep 09 08:41:47.656733 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.656736 2577 scope.go:117] "RemoveContainer" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" Sep 09 08:41:47.657053 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.657027 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17"} err="failed to get container status \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": rpc error: code = NotFound desc = could not find container \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": container with ID starting with 17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17 not found: ID does not exist" Sep 09 08:41:47.657053 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.657053 2577 scope.go:117] "RemoveContainer" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" Sep 09 08:41:47.657313 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.657293 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951"} err="failed to get container status \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": rpc error: code = NotFound desc = could not find container \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": container with ID starting with bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951 not found: ID does not exist" Sep 09 08:41:47.657386 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.657315 2577 scope.go:117] "RemoveContainer" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" Sep 09 08:41:47.657684 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.657659 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d"} err="failed to get container status \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": rpc error: code = NotFound desc = could not find container \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": container with ID starting with d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d not found: ID does not exist" Sep 09 08:41:47.657684 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.657685 2577 scope.go:117] "RemoveContainer" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" Sep 09 08:41:47.659670 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.658036 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e"} err="failed to get container status \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": rpc error: code = NotFound desc = could not find container \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": container with ID starting with 560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e not found: ID does not exist" Sep 09 08:41:47.659670 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.658060 2577 scope.go:117] "RemoveContainer" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" Sep 09 08:41:47.660076 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.660056 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7"} err="failed to get container status \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": rpc error: code = NotFound desc = could not find container \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": container with ID starting with 50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7 not found: ID does not exist" Sep 09 08:41:47.660076 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.660076 2577 scope.go:117] "RemoveContainer" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" Sep 09 08:41:47.660384 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.660354 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132"} err="failed to get container status \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": rpc error: code = NotFound desc = could not find container \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": container with ID starting with d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132 not found: ID does not exist" Sep 09 08:41:47.660384 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.660381 2577 scope.go:117] "RemoveContainer" containerID="b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d" Sep 09 08:41:47.660681 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.660660 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d"} err="failed to get container status \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": rpc error: code = NotFound desc = could not find container \"b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d\": container with ID starting with b97d4860c8ca7dfd3238b553a65463f0b964afd54598b029cf5a42f519d9283d not found: ID does not exist" Sep 09 08:41:47.660766 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.660682 2577 scope.go:117] "RemoveContainer" containerID="17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17" Sep 09 08:41:47.660823 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.660802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.661231 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.661002 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17"} err="failed to get container status \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": rpc error: code = NotFound desc = could not find container \"17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17\": container with ID starting with 17dc0f79d4fb23ec4beb8f7f28bfa9e51ac9e4f9d0189e6bb5f6c7598775af17 not found: ID does not exist" Sep 09 08:41:47.661231 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.661030 2577 scope.go:117] "RemoveContainer" containerID="bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951" Sep 09 08:41:47.661438 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.661414 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951"} err="failed to get container status \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": rpc error: code = NotFound desc = could not find container \"bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951\": container with ID starting with bd373bdf5735b2cfd6be870614d37f4c8bdb534f97d388bbf5b24775685d4951 not found: ID does not exist" Sep 09 08:41:47.661515 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.661440 2577 scope.go:117] "RemoveContainer" containerID="d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d" Sep 09 08:41:47.661704 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.661686 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d"} err="failed to get container status \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": rpc error: code = NotFound desc = could not find container \"d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d\": container with ID starting with d8a01e7b78c7419c597c905a01c7c1568f048dd82f7d6b4c8a30a8197e55e33d not found: ID does not exist" Sep 09 08:41:47.661756 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.661705 2577 scope.go:117] "RemoveContainer" containerID="560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e" Sep 09 08:41:47.661955 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.661932 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e"} err="failed to get container status \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": rpc error: code = NotFound desc = could not find container \"560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e\": container with ID starting with 560bb4a7cee9e3efd9c3309f234821678a618a44d3493e745401c0e0347d652e not found: ID does not exist" Sep 09 08:41:47.662039 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.661955 2577 scope.go:117] "RemoveContainer" containerID="50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7" Sep 09 08:41:47.662257 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.662239 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7"} err="failed to get container status \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": rpc error: code = NotFound desc = could not find container \"50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7\": container with ID starting with 50983023b62a3dbd89dd5dc9d2791046adda883c806d05a51a0653b14beef2a7 not found: ID does not exist" Sep 09 08:41:47.662330 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.662259 2577 scope.go:117] "RemoveContainer" containerID="d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132" Sep 09 08:41:47.662534 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.662517 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132"} err="failed to get container status \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": rpc error: code = NotFound desc = could not find container \"d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132\": container with ID starting with d460daa614c67f1826c80e02324c12ecca6b783d7cc96ca2a69321728ffc8132 not found: ID does not exist" Sep 09 08:41:47.663418 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.663400 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 08:41:47.663514 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.663400 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 08:41:47.663514 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.663427 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 08:41:47.663514 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.663400 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 08:41:47.663913 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.663856 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 08:41:47.663913 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.663894 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-703h3cils70l3\"" Sep 09 08:41:47.663913 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.663907 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 08:41:47.664141 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.663921 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 08:41:47.664305 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.664281 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 08:41:47.664405 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.664313 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zxmk7\"" Sep 09 08:41:47.664488 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.664438 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 08:41:47.664896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.664876 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 08:41:47.665030 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.664876 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 08:41:47.665030 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.665004 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 08:41:47.666959 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.666939 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 08:41:47.668934 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.668916 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 08:41:47.673240 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.673220 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:41:47.726662 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.726631 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.726662 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.726661 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6lgh\" (UniqueName: \"kubernetes.io/projected/177f7594-7af0-4d01-b453-4a15988ff6ca-kube-api-access-v6lgh\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.726833 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.726680 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.726833 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.726717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/177f7594-7af0-4d01-b453-4a15988ff6ca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.726833 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.726789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.726833 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.726823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727054 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.726851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727054 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.726929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727054 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727022 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/177f7594-7af0-4d01-b453-4a15988ff6ca-config-out\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727212 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727099 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727212 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727127 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727212 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727156 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727366 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727366 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-config\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727366 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727481 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727481 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727481 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727443 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-web-config\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727583 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727496 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.727583 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.727526 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.828869 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.828841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/177f7594-7af0-4d01-b453-4a15988ff6ca-config-out\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829001 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.828877 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829001 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.828895 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829078 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829078 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829046 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829175 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-config\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829175 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829175 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829124 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829175 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829160 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-web-config\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6lgh\" (UniqueName: \"kubernetes.io/projected/177f7594-7af0-4d01-b453-4a15988ff6ca-kube-api-access-v6lgh\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829370 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829335 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/177f7594-7af0-4d01-b453-4a15988ff6ca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829477 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829678 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829636 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.829962 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.829732 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.830177 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.830149 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.830652 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.830627 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.832770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.832090 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/177f7594-7af0-4d01-b453-4a15988ff6ca-config-out\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.832770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.832109 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/177f7594-7af0-4d01-b453-4a15988ff6ca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.832770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.832255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.832770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.832267 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.832770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.832515 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.832770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.832673 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.832770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.832682 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.832770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.832689 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.833777 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.833729 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.833876 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.833864 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-config\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.834168 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.834148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.834266 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.834240 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.834704 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.834677 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/177f7594-7af0-4d01-b453-4a15988ff6ca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.834859 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.834841 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-web-config\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.835232 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.835216 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/177f7594-7af0-4d01-b453-4a15988ff6ca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.840393 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.840376 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6lgh\" (UniqueName: \"kubernetes.io/projected/177f7594-7af0-4d01-b453-4a15988ff6ca-kube-api-access-v6lgh\") pod \"prometheus-k8s-0\" (UID: \"177f7594-7af0-4d01-b453-4a15988ff6ca\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:47.970756 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:47.970688 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:41:48.250781 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:48.250751 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 08:41:48.251182 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:41:48.251163 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177f7594_7af0_4d01_b453_4a15988ff6ca.slice/crio-111ea316a5bd84840310fd977974253b0f5bf68de47e9e0836f8b9fbef739734 WatchSource:0}: Error finding container 111ea316a5bd84840310fd977974253b0f5bf68de47e9e0836f8b9fbef739734: Status 404 returned error can't find the container with id 111ea316a5bd84840310fd977974253b0f5bf68de47e9e0836f8b9fbef739734 Sep 09 08:41:48.604983 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:48.604889 2577 generic.go:356] "Generic (PLEG): container finished" podID="177f7594-7af0-4d01-b453-4a15988ff6ca" containerID="294f46dc2d2a540342eb2088d85edff851eaac7ae7e569606907e64004e20822" exitCode=0 Sep 09 08:41:48.604983 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:48.604961 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"177f7594-7af0-4d01-b453-4a15988ff6ca","Type":"ContainerDied","Data":"294f46dc2d2a540342eb2088d85edff851eaac7ae7e569606907e64004e20822"} Sep 09 08:41:48.605375 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:48.605037 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"177f7594-7af0-4d01-b453-4a15988ff6ca","Type":"ContainerStarted","Data":"111ea316a5bd84840310fd977974253b0f5bf68de47e9e0836f8b9fbef739734"} Sep 09 08:41:48.635541 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:48.635514 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8caae4fa-18d8-4b09-88a8-e787efe60fbe" path="/var/lib/kubelet/pods/8caae4fa-18d8-4b09-88a8-e787efe60fbe/volumes" Sep 09 08:41:49.616153 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:49.616117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"177f7594-7af0-4d01-b453-4a15988ff6ca","Type":"ContainerStarted","Data":"2ff3c4de0bcb9e90381e0fbaaa7395235469f6c08cb608f1c3fdea2725a3f5f9"} Sep 09 08:41:49.616520 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:49.616199 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"177f7594-7af0-4d01-b453-4a15988ff6ca","Type":"ContainerStarted","Data":"9c4f729430da343d1e8bf4d9a00a939fe0d499bcbec0904712831566e0c5670b"} Sep 09 08:41:49.616520 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:49.616246 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"177f7594-7af0-4d01-b453-4a15988ff6ca","Type":"ContainerStarted","Data":"4b08aa4bfcf08c3f612c328f9cdde187c1eb1fbbcca946e406341f9c81ccd569"} Sep 09 08:41:49.616520 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:49.616282 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"177f7594-7af0-4d01-b453-4a15988ff6ca","Type":"ContainerStarted","Data":"b0312c07be51d3bfb9a0089562c79b3a10e27263ff6d6828bca38ff86200beda"} Sep 09 08:41:49.616520 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:49.616317 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"177f7594-7af0-4d01-b453-4a15988ff6ca","Type":"ContainerStarted","Data":"892aa0aade0ad7599c4e7ed6b3339e3617ce9df6e21bac282b4417baf93d333b"} Sep 09 08:41:49.616520 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:49.616353 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"177f7594-7af0-4d01-b453-4a15988ff6ca","Type":"ContainerStarted","Data":"dbf14a880d26cb4fe7e8e857f3a1122491a0f0ce4ce10d695e83321a0c5b52bf"} Sep 09 08:41:49.645729 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:49.645450 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.62223959 podStartE2EDuration="2.64542963s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.19004ms" isStatefulPod=false podCreationTimestamp="2026-09-09 08:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:41:49.643674681 +0000 UTC m=+153.477038257" watchObservedRunningTime="2026-09-09 08:41:49.64542963 +0000 UTC m=+153.478793188" Sep 09 08:41:52.971122 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:41:52.971090 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:42:00.319785 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:00.319751 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c4cb6d6d-c44dz"] Sep 09 08:42:25.339760 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.339697 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c4cb6d6d-c44dz" podUID="e82ce7cc-1e71-42e7-9a66-30839d1f11c0" containerName="console" containerID="cri-o://ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9" gracePeriod=15 Sep 09 08:42:25.576907 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.576883 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:42:25.643883 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.643788 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-service-ca\") pod \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " Sep 09 08:42:25.643883 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.643835 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-trusted-ca-bundle\") pod \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " Sep 09 08:42:25.643883 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.643854 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-oauth-serving-cert\") pod \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " Sep 09 08:42:25.644151 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.643898 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-config\") pod \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " Sep 09 08:42:25.644151 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.643922 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t6zg\" (UniqueName: \"kubernetes.io/projected/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-kube-api-access-5t6zg\") pod \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " Sep 09 08:42:25.644151 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.643946 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-oauth-config\") pod \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " Sep 09 08:42:25.644151 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.644012 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-serving-cert\") pod \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\" (UID: \"e82ce7cc-1e71-42e7-9a66-30839d1f11c0\") " Sep 09 08:42:25.644366 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.644319 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-service-ca" (OuterVolumeSpecName: "service-ca") pod "e82ce7cc-1e71-42e7-9a66-30839d1f11c0" (UID: "e82ce7cc-1e71-42e7-9a66-30839d1f11c0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:42:25.644366 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.644331 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e82ce7cc-1e71-42e7-9a66-30839d1f11c0" (UID: "e82ce7cc-1e71-42e7-9a66-30839d1f11c0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:42:25.644475 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.644370 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e82ce7cc-1e71-42e7-9a66-30839d1f11c0" (UID: "e82ce7cc-1e71-42e7-9a66-30839d1f11c0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:42:25.644475 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.644408 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-config" (OuterVolumeSpecName: "console-config") pod "e82ce7cc-1e71-42e7-9a66-30839d1f11c0" (UID: "e82ce7cc-1e71-42e7-9a66-30839d1f11c0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:42:25.646243 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.646217 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e82ce7cc-1e71-42e7-9a66-30839d1f11c0" (UID: "e82ce7cc-1e71-42e7-9a66-30839d1f11c0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:42:25.646338 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.646234 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e82ce7cc-1e71-42e7-9a66-30839d1f11c0" (UID: "e82ce7cc-1e71-42e7-9a66-30839d1f11c0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:42:25.646338 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.646256 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-kube-api-access-5t6zg" (OuterVolumeSpecName: "kube-api-access-5t6zg") pod "e82ce7cc-1e71-42e7-9a66-30839d1f11c0" (UID: "e82ce7cc-1e71-42e7-9a66-30839d1f11c0"). InnerVolumeSpecName "kube-api-access-5t6zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:42:25.745518 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.745484 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-serving-cert\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:42:25.745518 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.745511 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-service-ca\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:42:25.745518 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.745521 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-trusted-ca-bundle\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:42:25.745737 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.745530 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-oauth-serving-cert\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:42:25.745737 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.745538 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-config\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:42:25.745737 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.745547 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t6zg\" (UniqueName: \"kubernetes.io/projected/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-kube-api-access-5t6zg\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:42:25.745737 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.745555 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e82ce7cc-1e71-42e7-9a66-30839d1f11c0-console-oauth-config\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:42:25.767080 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.767048 2577 generic.go:356] "Generic (PLEG): container finished" podID="e82ce7cc-1e71-42e7-9a66-30839d1f11c0" containerID="ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9" exitCode=0 Sep 09 08:42:25.767196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.767093 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4cb6d6d-c44dz" Sep 09 08:42:25.767196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.767133 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4cb6d6d-c44dz" event={"ID":"e82ce7cc-1e71-42e7-9a66-30839d1f11c0","Type":"ContainerDied","Data":"ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9"} Sep 09 08:42:25.767196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.767184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4cb6d6d-c44dz" event={"ID":"e82ce7cc-1e71-42e7-9a66-30839d1f11c0","Type":"ContainerDied","Data":"1832797f3f1b491ca3fadeb529262497d938cab440745ddf70cc27bb72e57c93"} Sep 09 08:42:25.767317 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.767204 2577 scope.go:117] "RemoveContainer" containerID="ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9" Sep 09 08:42:25.776014 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.775996 2577 scope.go:117] "RemoveContainer" containerID="ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9" Sep 09 08:42:25.776274 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:42:25.776257 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9\": container with ID starting with ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9 not found: ID does not exist" containerID="ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9" Sep 09 08:42:25.776320 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.776282 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9"} err="failed to get container status \"ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9\": rpc error: code = NotFound desc = could not find container \"ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9\": container with ID starting with ef2d2e1e9f8bac5557e5e9f200cbaf20bd1ed26c1ca939c966bbde31b7299ea9 not found: ID does not exist" Sep 09 08:42:25.787655 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.787634 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c4cb6d6d-c44dz"] Sep 09 08:42:25.790765 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:25.790744 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c4cb6d6d-c44dz"] Sep 09 08:42:26.633050 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:26.633013 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e82ce7cc-1e71-42e7-9a66-30839d1f11c0" path="/var/lib/kubelet/pods/e82ce7cc-1e71-42e7-9a66-30839d1f11c0/volumes" Sep 09 08:42:47.971720 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:47.971687 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:42:47.987085 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:47.987063 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:42:48.878411 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:42:48.878381 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 08:44:16.528896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:16.528872 2577 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 08:44:30.218794 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.218758 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m"] Sep 09 08:44:30.221085 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.219150 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e82ce7cc-1e71-42e7-9a66-30839d1f11c0" containerName="console" Sep 09 08:44:30.221085 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.219163 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e82ce7cc-1e71-42e7-9a66-30839d1f11c0" containerName="console" Sep 09 08:44:30.221085 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.219220 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e82ce7cc-1e71-42e7-9a66-30839d1f11c0" containerName="console" Sep 09 08:44:30.222336 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.222322 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.224668 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.224643 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 09 08:44:30.224668 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.224660 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 09 08:44:30.224901 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.224887 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 09 08:44:30.224968 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.224949 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-fhbw4\"" Sep 09 08:44:30.225129 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.225111 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 09 08:44:30.225171 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.225112 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 09 08:44:30.225218 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.225185 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 09 08:44:30.225361 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.225347 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 09 08:44:30.235413 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.235392 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m"] Sep 09 08:44:30.305101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.305068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.305101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.305105 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a041d277-298d-4513-85fa-0652837cd3f6-feast-data\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.305300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.305131 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.305300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.305175 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4x22\" (UniqueName: \"kubernetes.io/projected/a041d277-298d-4513-85fa-0652837cd3f6-kube-api-access-p4x22\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.305300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.305263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.305398 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.305325 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.406368 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.406334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.406368 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.406376 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.406594 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.406417 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.406594 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.406448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a041d277-298d-4513-85fa-0652837cd3f6-feast-data\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.406594 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.406484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.406594 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.406512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4x22\" (UniqueName: \"kubernetes.io/projected/a041d277-298d-4513-85fa-0652837cd3f6-kube-api-access-p4x22\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.406594 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:44:30.406485 2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 09 08:44:30.406764 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:44:30.406562 2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 09 08:44:30.406764 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:44:30.406663 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls podName:a041d277-298d-4513-85fa-0652837cd3f6 nodeName:}" failed. No retries permitted until 2026-09-09 08:44:30.906641385 +0000 UTC m=+314.740004938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls") pod "feast-simple-feast-setup-787c77f4b9-rqg5m" (UID: "a041d277-298d-4513-85fa-0652837cd3f6") : secret "feast-simple-feast-setup-registry-tls" not found Sep 09 08:44:30.406764 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:44:30.406697 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls podName:a041d277-298d-4513-85fa-0652837cd3f6 nodeName:}" failed. No retries permitted until 2026-09-09 08:44:30.906680793 +0000 UTC m=+314.740044331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls") pod "feast-simple-feast-setup-787c77f4b9-rqg5m" (UID: "a041d277-298d-4513-85fa-0652837cd3f6") : secret "feast-simple-feast-setup-offline-tls" not found Sep 09 08:44:30.406764 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:44:30.406562 2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 09 08:44:30.406764 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:44:30.406753 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls podName:a041d277-298d-4513-85fa-0652837cd3f6 nodeName:}" failed. No retries permitted until 2026-09-09 08:44:30.906745191 +0000 UTC m=+314.740108734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls") pod "feast-simple-feast-setup-787c77f4b9-rqg5m" (UID: "a041d277-298d-4513-85fa-0652837cd3f6") : secret "feast-simple-feast-setup-ui-tls" not found Sep 09 08:44:30.406764 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:44:30.406588 2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 09 08:44:30.407030 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:44:30.406796 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls podName:a041d277-298d-4513-85fa-0652837cd3f6 nodeName:}" failed. No retries permitted until 2026-09-09 08:44:30.906786462 +0000 UTC m=+314.740150006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls") pod "feast-simple-feast-setup-787c77f4b9-rqg5m" (UID: "a041d277-298d-4513-85fa-0652837cd3f6") : secret "feast-simple-feast-setup-online-tls" not found Sep 09 08:44:30.407030 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.406858 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a041d277-298d-4513-85fa-0652837cd3f6-feast-data\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.414957 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.414933 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4x22\" (UniqueName: \"kubernetes.io/projected/a041d277-298d-4513-85fa-0652837cd3f6-kube-api-access-p4x22\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.911775 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.911729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.911775 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.911781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.912032 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.911827 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.912032 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.911858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.914287 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.914262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.914410 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.914301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.914410 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.914335 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:30.914410 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:30.914358 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls\") pod \"feast-simple-feast-setup-787c77f4b9-rqg5m\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:31.143101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:31.143045 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:44:31.417006 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:31.416960 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m"] Sep 09 08:44:31.419593 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:44:31.419564 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda041d277_298d_4513_85fa_0652837cd3f6.slice/crio-0069a605e9273f170ff05c4cb9a39017d367045f6e6aa9fe57d6bf729eab3316 WatchSource:0}: Error finding container 0069a605e9273f170ff05c4cb9a39017d367045f6e6aa9fe57d6bf729eab3316: Status 404 returned error can't find the container with id 0069a605e9273f170ff05c4cb9a39017d367045f6e6aa9fe57d6bf729eab3316 Sep 09 08:44:31.421366 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:31.421350 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 08:44:32.287128 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:32.287090 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerStarted","Data":"0069a605e9273f170ff05c4cb9a39017d367045f6e6aa9fe57d6bf729eab3316"} Sep 09 08:44:55.407582 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:44:55.407550 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerStarted","Data":"5ea326aeb9336022755f48cd711467db24209031eab19d65608437760f5e6991"} Sep 09 08:45:00.429919 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:00.429887 2577 generic.go:356] "Generic (PLEG): container finished" podID="a041d277-298d-4513-85fa-0652837cd3f6" containerID="5ea326aeb9336022755f48cd711467db24209031eab19d65608437760f5e6991" exitCode=0 Sep 09 08:45:00.430355 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:00.429949 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerDied","Data":"5ea326aeb9336022755f48cd711467db24209031eab19d65608437760f5e6991"} Sep 09 08:45:01.437055 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:01.437011 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerStarted","Data":"95e25c1d9f540edfaaf8863a1ccfdda1a3e44b7a97964e8eb368f39a23301c31"} Sep 09 08:45:06.461830 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:06.461789 2577 generic.go:356] "Generic (PLEG): container finished" podID="a041d277-298d-4513-85fa-0652837cd3f6" containerID="95e25c1d9f540edfaaf8863a1ccfdda1a3e44b7a97964e8eb368f39a23301c31" exitCode=0 Sep 09 08:45:06.462219 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:06.461862 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerDied","Data":"95e25c1d9f540edfaaf8863a1ccfdda1a3e44b7a97964e8eb368f39a23301c31"} Sep 09 08:45:07.473339 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:07.473291 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerStarted","Data":"b768d1e13db801191df69e13cdca94d06db2e0ed4ce7db827c5803384de3fc3a"} Sep 09 08:45:07.473814 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:07.473366 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerStarted","Data":"45a6f24bef80d7392f673d53c5854ded62d314bc19eef737fdd58b06e2baca02"} Sep 09 08:45:07.473814 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:07.473392 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerStarted","Data":"6cfa602547f739b71d1471bc081b7915503a831c1dd2fd4b583396f8c5dc0f46"} Sep 09 08:45:07.473814 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:07.473412 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerStarted","Data":"766c73e1a3550e068f769d6e223ce2bb64c5470345cea5809c05fc792d1409ac"} Sep 09 08:45:07.499659 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:07.499585 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podStartSLOduration=13.602147087 podStartE2EDuration="37.499558969s" totalImagesPullingTime="23.897411882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:45:07.496865317 +0000 UTC m=+351.330228883" watchObservedRunningTime="2026-09-09 08:45:07.499558969 +0000 UTC m=+351.332922530" Sep 09 08:45:10.144320 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:10.144030 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:10.144320 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:10.144098 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:10.144320 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:10.144118 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:10.144320 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:10.144240 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:10.146314 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:10.146245 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 09 08:45:10.146468 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:10.146319 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 09 08:45:10.146468 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:10.146281 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 09 08:45:10.146468 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:10.146270 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 09 08:45:11.143984 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:11.143915 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:11.144190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:11.144004 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:11.144190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:11.144024 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:11.144190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:11.144042 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:13.143692 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:13.143618 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 09 08:45:13.144316 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:13.143616 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 09 08:45:13.144316 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:13.143616 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 09 08:45:13.144316 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:13.143736 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 09 08:45:16.144912 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:16.144880 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:16.145348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:16.145062 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:16.145348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:16.145223 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:16.145754 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:16.145730 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:16.145860 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:16.145805 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:16.145860 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:16.145856 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:16.149939 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:16.149922 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:45:16.529099 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:45:16.529072 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:46:18.426598 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.426563 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m"] Sep 09 08:46:18.427139 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.426830 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="registry" containerID="cri-o://766c73e1a3550e068f769d6e223ce2bb64c5470345cea5809c05fc792d1409ac" gracePeriod=30 Sep 09 08:46:18.427139 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.426911 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" containerID="cri-o://b768d1e13db801191df69e13cdca94d06db2e0ed4ce7db827c5803384de3fc3a" gracePeriod=30 Sep 09 08:46:18.427139 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.427018 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" containerID="cri-o://6cfa602547f739b71d1471bc081b7915503a831c1dd2fd4b583396f8c5dc0f46" gracePeriod=30 Sep 09 08:46:18.427139 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.427118 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="offline" containerID="cri-o://45a6f24bef80d7392f673d53c5854ded62d314bc19eef737fdd58b06e2baca02" gracePeriod=30 Sep 09 08:46:18.809723 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.809680 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl"] Sep 09 08:46:18.813949 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.813927 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:18.816571 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.816552 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-7xdns\"" Sep 09 08:46:18.824451 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.824428 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl"] Sep 09 08:46:18.955407 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.955369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-online-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:18.955584 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.955424 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:18.955584 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.955444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:18.955584 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.955495 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1c4771d2-77d1-43e4-82c0-9a89fccebffc-feast-data\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:18.955584 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.955546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:18.955584 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:18.955577 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-794l8\" (UniqueName: \"kubernetes.io/projected/1c4771d2-77d1-43e4-82c0-9a89fccebffc-kube-api-access-794l8\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.056217 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.056182 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-online-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.056348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.056244 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.056348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.056261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.056348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.056285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1c4771d2-77d1-43e4-82c0-9a89fccebffc-feast-data\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.056348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.056309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.056348 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.056337 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-794l8\" (UniqueName: \"kubernetes.io/projected/1c4771d2-77d1-43e4-82c0-9a89fccebffc-kube-api-access-794l8\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.056606 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:46:19.056396 2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 09 08:46:19.056606 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:46:19.056404 2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 09 08:46:19.056606 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:46:19.056476 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls podName:1c4771d2-77d1-43e4-82c0-9a89fccebffc nodeName:}" failed. No retries permitted until 2026-09-09 08:46:19.556455737 +0000 UTC m=+423.389819290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls") pod "feast-simple-feast-setup-787c77f4b9-q9ggl" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc") : secret "feast-simple-feast-setup-offline-tls" not found Sep 09 08:46:19.056606 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:46:19.056493 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls podName:1c4771d2-77d1-43e4-82c0-9a89fccebffc nodeName:}" failed. No retries permitted until 2026-09-09 08:46:19.55648588 +0000 UTC m=+423.389849420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls") pod "feast-simple-feast-setup-787c77f4b9-q9ggl" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc") : secret "feast-simple-feast-setup-ui-tls" not found Sep 09 08:46:19.056606 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:46:19.056524 2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 09 08:46:19.056606 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:46:19.056565 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls podName:1c4771d2-77d1-43e4-82c0-9a89fccebffc nodeName:}" failed. No retries permitted until 2026-09-09 08:46:19.556548166 +0000 UTC m=+423.389911702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls") pod "feast-simple-feast-setup-787c77f4b9-q9ggl" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc") : secret "feast-simple-feast-setup-registry-tls" not found Sep 09 08:46:19.056962 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.056660 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1c4771d2-77d1-43e4-82c0-9a89fccebffc-feast-data\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.058899 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.058874 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-online-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.067025 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.067000 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-794l8\" (UniqueName: \"kubernetes.io/projected/1c4771d2-77d1-43e4-82c0-9a89fccebffc-kube-api-access-794l8\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.560847 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.560813 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.560847 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.560849 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.561296 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.560871 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.563374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.563340 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.563374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.563362 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.563526 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.563399 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls\") pod \"feast-simple-feast-setup-787c77f4b9-q9ggl\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.724711 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.724663 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:19.792633 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.792605 2577 generic.go:356] "Generic (PLEG): container finished" podID="a041d277-298d-4513-85fa-0652837cd3f6" containerID="b768d1e13db801191df69e13cdca94d06db2e0ed4ce7db827c5803384de3fc3a" exitCode=0 Sep 09 08:46:19.792633 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.792629 2577 generic.go:356] "Generic (PLEG): container finished" podID="a041d277-298d-4513-85fa-0652837cd3f6" containerID="6cfa602547f739b71d1471bc081b7915503a831c1dd2fd4b583396f8c5dc0f46" exitCode=0 Sep 09 08:46:19.792812 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.792676 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerDied","Data":"b768d1e13db801191df69e13cdca94d06db2e0ed4ce7db827c5803384de3fc3a"} Sep 09 08:46:19.792812 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.792716 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerDied","Data":"6cfa602547f739b71d1471bc081b7915503a831c1dd2fd4b583396f8c5dc0f46"} Sep 09 08:46:19.998442 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:19.998412 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl"] Sep 09 08:46:20.001308 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:46:20.001285 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c4771d2_77d1_43e4_82c0_9a89fccebffc.slice/crio-f883ae07e54af4c5b5f2050ed65c53bbf7022fbd5d19b1d4d1847cb7d85baf4c WatchSource:0}: Error finding container f883ae07e54af4c5b5f2050ed65c53bbf7022fbd5d19b1d4d1847cb7d85baf4c: Status 404 returned error can't find the container with id f883ae07e54af4c5b5f2050ed65c53bbf7022fbd5d19b1d4d1847cb7d85baf4c Sep 09 08:46:20.798436 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:20.798400 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerStarted","Data":"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60"} Sep 09 08:46:20.798806 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:20.798443 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerStarted","Data":"f883ae07e54af4c5b5f2050ed65c53bbf7022fbd5d19b1d4d1847cb7d85baf4c"} Sep 09 08:46:24.817808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:24.817776 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerID="99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60" exitCode=0 Sep 09 08:46:24.818187 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:24.817852 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerDied","Data":"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60"} Sep 09 08:46:25.824013 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:25.823956 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerStarted","Data":"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0"} Sep 09 08:46:26.145931 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:26.145833 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 09 08:46:26.526661 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:26.526608 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 09 08:46:30.848061 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:30.847954 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerID="71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0" exitCode=0 Sep 09 08:46:30.848061 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:30.848038 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerDied","Data":"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0"} Sep 09 08:46:31.859209 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:31.859166 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerStarted","Data":"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff"} Sep 09 08:46:31.859677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:31.859232 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerStarted","Data":"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f"} Sep 09 08:46:31.859677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:31.859253 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerStarted","Data":"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121"} Sep 09 08:46:31.859677 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:31.859271 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerStarted","Data":"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b"} Sep 09 08:46:31.887693 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:31.887628 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podStartSLOduration=13.887606137 podStartE2EDuration="13.887606137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:46:31.886494878 +0000 UTC m=+435.719858438" watchObservedRunningTime="2026-09-09 08:46:31.887606137 +0000 UTC m=+435.720969699" Sep 09 08:46:34.725849 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:34.725796 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:34.726367 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:34.725876 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:34.726367 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:34.725897 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:34.726367 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:34.725917 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:34.728228 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:34.728189 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 09 08:46:34.728228 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:34.728154 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 09 08:46:34.728429 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:34.728194 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused" Sep 09 08:46:34.728429 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:34.728190 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Sep 09 08:46:36.146144 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:36.146092 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 09 08:46:36.526858 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:36.526810 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 09 08:46:37.725151 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:37.725096 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused" Sep 09 08:46:37.725651 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:37.725196 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Sep 09 08:46:37.725651 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:37.725266 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 09 08:46:37.725808 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:37.725119 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 09 08:46:39.725742 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:39.725648 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:39.725742 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:39.725711 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:39.725742 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:39.725725 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:39.725742 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:39.725741 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:40.725825 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:40.725790 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:40.726437 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:40.726418 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:40.726576 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:40.726562 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:40.726717 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:40.726705 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:40.731166 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:40.731141 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:40.913145 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:40.913113 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:40.913547 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:40.913526 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:40.915965 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:40.915944 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:46:46.146467 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:46.146427 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 09 08:46:46.146909 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:46.146562 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:46:46.526248 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:46.526207 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 09 08:46:46.526439 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:46.526341 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:46:48.949585 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:48.949542 2577 generic.go:356] "Generic (PLEG): container finished" podID="a041d277-298d-4513-85fa-0652837cd3f6" containerID="45a6f24bef80d7392f673d53c5854ded62d314bc19eef737fdd58b06e2baca02" exitCode=137 Sep 09 08:46:48.949585 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:48.949575 2577 generic.go:356] "Generic (PLEG): container finished" podID="a041d277-298d-4513-85fa-0652837cd3f6" containerID="766c73e1a3550e068f769d6e223ce2bb64c5470345cea5809c05fc792d1409ac" exitCode=137 Sep 09 08:46:48.950117 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:48.949606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerDied","Data":"45a6f24bef80d7392f673d53c5854ded62d314bc19eef737fdd58b06e2baca02"} Sep 09 08:46:48.950117 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:48.949648 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerDied","Data":"766c73e1a3550e068f769d6e223ce2bb64c5470345cea5809c05fc792d1409ac"} Sep 09 08:46:49.072451 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.072428 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:46:49.119992 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.119947 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls\") pod \"a041d277-298d-4513-85fa-0652837cd3f6\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " Sep 09 08:46:49.120138 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.120002 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4x22\" (UniqueName: \"kubernetes.io/projected/a041d277-298d-4513-85fa-0652837cd3f6-kube-api-access-p4x22\") pod \"a041d277-298d-4513-85fa-0652837cd3f6\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " Sep 09 08:46:49.120138 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.120111 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls\") pod \"a041d277-298d-4513-85fa-0652837cd3f6\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " Sep 09 08:46:49.120260 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.120149 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls\") pod \"a041d277-298d-4513-85fa-0652837cd3f6\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " Sep 09 08:46:49.120260 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.120177 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls\") pod \"a041d277-298d-4513-85fa-0652837cd3f6\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " Sep 09 08:46:49.120260 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.120222 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a041d277-298d-4513-85fa-0652837cd3f6-feast-data\") pod \"a041d277-298d-4513-85fa-0652837cd3f6\" (UID: \"a041d277-298d-4513-85fa-0652837cd3f6\") " Sep 09 08:46:49.120907 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.120876 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a041d277-298d-4513-85fa-0652837cd3f6-feast-data" (OuterVolumeSpecName: "feast-data") pod "a041d277-298d-4513-85fa-0652837cd3f6" (UID: "a041d277-298d-4513-85fa-0652837cd3f6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:46:49.122302 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.122278 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a041d277-298d-4513-85fa-0652837cd3f6" (UID: "a041d277-298d-4513-85fa-0652837cd3f6"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:46:49.122406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.122362 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls" (OuterVolumeSpecName: "online-tls") pod "a041d277-298d-4513-85fa-0652837cd3f6" (UID: "a041d277-298d-4513-85fa-0652837cd3f6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:46:49.122406 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.122375 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a041d277-298d-4513-85fa-0652837cd3f6" (UID: "a041d277-298d-4513-85fa-0652837cd3f6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:46:49.122591 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.122570 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a041d277-298d-4513-85fa-0652837cd3f6-kube-api-access-p4x22" (OuterVolumeSpecName: "kube-api-access-p4x22") pod "a041d277-298d-4513-85fa-0652837cd3f6" (UID: "a041d277-298d-4513-85fa-0652837cd3f6"). InnerVolumeSpecName "kube-api-access-p4x22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:46:49.122751 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.122728 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a041d277-298d-4513-85fa-0652837cd3f6" (UID: "a041d277-298d-4513-85fa-0652837cd3f6"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:46:49.221293 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.221230 2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-ui-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:46:49.221293 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.221252 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-online-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:46:49.221293 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.221262 2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-offline-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:46:49.221293 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.221269 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a041d277-298d-4513-85fa-0652837cd3f6-feast-data\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:46:49.221293 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.221277 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a041d277-298d-4513-85fa-0652837cd3f6-registry-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:46:49.221293 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.221284 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4x22\" (UniqueName: \"kubernetes.io/projected/a041d277-298d-4513-85fa-0652837cd3f6-kube-api-access-p4x22\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:46:49.957476 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.957431 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" event={"ID":"a041d277-298d-4513-85fa-0652837cd3f6","Type":"ContainerDied","Data":"0069a605e9273f170ff05c4cb9a39017d367045f6e6aa9fe57d6bf729eab3316"} Sep 09 08:46:49.957949 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.957491 2577 scope.go:117] "RemoveContainer" containerID="b768d1e13db801191df69e13cdca94d06db2e0ed4ce7db827c5803384de3fc3a" Sep 09 08:46:49.957949 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.957446 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m" Sep 09 08:46:49.966075 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.966056 2577 scope.go:117] "RemoveContainer" containerID="45a6f24bef80d7392f673d53c5854ded62d314bc19eef737fdd58b06e2baca02" Sep 09 08:46:49.973435 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.973417 2577 scope.go:117] "RemoveContainer" containerID="6cfa602547f739b71d1471bc081b7915503a831c1dd2fd4b583396f8c5dc0f46" Sep 09 08:46:49.979350 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.979329 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m"] Sep 09 08:46:49.981740 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.981716 2577 scope.go:117] "RemoveContainer" containerID="766c73e1a3550e068f769d6e223ce2bb64c5470345cea5809c05fc792d1409ac" Sep 09 08:46:49.983349 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.983331 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-rqg5m"] Sep 09 08:46:49.988929 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.988914 2577 scope.go:117] "RemoveContainer" containerID="95e25c1d9f540edfaaf8863a1ccfdda1a3e44b7a97964e8eb368f39a23301c31" Sep 09 08:46:49.995876 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:49.995861 2577 scope.go:117] "RemoveContainer" containerID="5ea326aeb9336022755f48cd711467db24209031eab19d65608437760f5e6991" Sep 09 08:46:50.633938 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:46:50.633909 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a041d277-298d-4513-85fa-0652837cd3f6" path="/var/lib/kubelet/pods/a041d277-298d-4513-85fa-0652837cd3f6/volumes" Sep 09 08:47:43.791813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.791776 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf"] Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792130 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="offline" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792143 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="offline" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792157 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="feast-apply" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792162 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="feast-apply" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792170 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="feast-init" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792175 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="feast-init" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792185 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="registry" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792191 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="registry" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792198 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792203 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792211 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792216 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792271 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="offline" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792279 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="online" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792287 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="registry" Sep 09 08:47:43.792300 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.792292 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a041d277-298d-4513-85fa-0652837cd3f6" containerName="ui" Sep 09 08:47:43.795505 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.795489 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.798031 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.798012 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 09 08:47:43.798158 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.798135 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 09 08:47:43.798269 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.798172 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 09 08:47:43.798269 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.798220 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 09 08:47:43.798383 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.798267 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 09 08:47:43.798383 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.798308 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 09 08:47:43.798514 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.798494 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 09 08:47:43.799463 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.799444 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-fd8dm\"" Sep 09 08:47:43.806341 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.806322 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf"] Sep 09 08:47:43.864315 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.864282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-online-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.864462 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.864352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dg7j\" (UniqueName: \"kubernetes.io/projected/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-kube-api-access-7dg7j\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.864462 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.864382 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.864462 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.864396 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-feast-data\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.864462 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.864455 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-registry-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.864635 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.864481 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-offline-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.965184 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.965148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-offline-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.965382 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.965213 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-online-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.965382 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.965274 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dg7j\" (UniqueName: \"kubernetes.io/projected/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-kube-api-access-7dg7j\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.965382 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.965314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.965382 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.965334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-feast-data\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.965382 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.965373 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-registry-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.965644 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:47:43.965450 2577 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 09 08:47:43.965644 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:47:43.965642 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls podName:e9c895b4-ba55-49a5-97d6-5ae3402da5b1 nodeName:}" failed. No retries permitted until 2026-09-09 08:47:44.465529929 +0000 UTC m=+508.298893494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls") pod "feast-simple-feast-remote-setup-75bb5589c6-kcqvf" (UID: "e9c895b4-ba55-49a5-97d6-5ae3402da5b1") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 09 08:47:43.965772 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.965743 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-feast-data\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.966081 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.966059 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-registry-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.967613 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.967593 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-offline-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.967695 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.967628 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-online-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:43.976816 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:43.976792 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dg7j\" (UniqueName: \"kubernetes.io/projected/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-kube-api-access-7dg7j\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:44.470765 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:44.470735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:44.472958 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:44.472935 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls\") pod \"feast-simple-feast-remote-setup-75bb5589c6-kcqvf\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:44.704254 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:44.704221 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:47:44.981481 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:44.981454 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf"] Sep 09 08:47:44.982272 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:47:44.982240 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9c895b4_ba55_49a5_97d6_5ae3402da5b1.slice/crio-dbcbc0e6565efd7ef2f65bcb6bbc6d754404d8b4e5a103034cda17fb28f4edb2 WatchSource:0}: Error finding container dbcbc0e6565efd7ef2f65bcb6bbc6d754404d8b4e5a103034cda17fb28f4edb2: Status 404 returned error can't find the container with id dbcbc0e6565efd7ef2f65bcb6bbc6d754404d8b4e5a103034cda17fb28f4edb2 Sep 09 08:47:45.190893 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:45.190858 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerStarted","Data":"071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f"} Sep 09 08:47:45.191079 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:45.190911 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerStarted","Data":"dbcbc0e6565efd7ef2f65bcb6bbc6d754404d8b4e5a103034cda17fb28f4edb2"} Sep 09 08:47:50.212550 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:50.212516 2577 generic.go:356] "Generic (PLEG): container finished" podID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerID="071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f" exitCode=0 Sep 09 08:47:50.212910 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:50.212591 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerDied","Data":"071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f"} Sep 09 08:47:51.218864 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:47:51.218826 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerStarted","Data":"b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61"} Sep 09 08:48:11.307760 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:11.307727 2577 generic.go:356] "Generic (PLEG): container finished" podID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerID="b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61" exitCode=0 Sep 09 08:48:11.308203 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:11.307799 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerDied","Data":"b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61"} Sep 09 08:48:12.317055 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:12.317011 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerStarted","Data":"c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592"} Sep 09 08:48:12.317520 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:12.317083 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerStarted","Data":"a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c"} Sep 09 08:48:12.317520 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:12.317104 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerStarted","Data":"b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed"} Sep 09 08:48:12.341475 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:12.341410 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podStartSLOduration=29.341390658 podStartE2EDuration="29.341390658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:47:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:48:12.340802313 +0000 UTC m=+536.174165872" watchObservedRunningTime="2026-09-09 08:48:12.341390658 +0000 UTC m=+536.174754218" Sep 09 08:48:14.704335 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.704292 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:14.704887 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.704363 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:14.704887 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.704388 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:14.704887 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.704408 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:14.704887 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.704425 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:14.704887 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.704442 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:14.706382 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.706345 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 09 08:48:14.706382 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.706363 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 09 08:48:14.706536 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:14.706400 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 09 08:48:17.704535 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:17.704493 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 09 08:48:17.705145 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:17.704493 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 09 08:48:17.705264 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:17.704584 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 09 08:48:20.705952 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:20.705923 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:20.706361 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:20.706065 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:20.706748 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:20.706727 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:20.707057 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:20.707037 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:20.725920 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:20.725897 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:48:21.393259 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:48:21.393233 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:49:24.019593 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.019561 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf"] Sep 09 08:49:24.020106 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.019789 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" containerID="cri-o://b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed" gracePeriod=30 Sep 09 08:49:24.020106 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.019859 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="offline" containerID="cri-o://a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c" gracePeriod=30 Sep 09 08:49:24.020106 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.019865 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" containerID="cri-o://c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592" gracePeriod=30 Sep 09 08:49:24.105853 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.105812 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl"] Sep 09 08:49:24.106176 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.106124 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="registry" containerID="cri-o://4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b" gracePeriod=30 Sep 09 08:49:24.106280 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.106185 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="offline" containerID="cri-o://50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f" gracePeriod=30 Sep 09 08:49:24.106355 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.106198 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" containerID="cri-o://0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121" gracePeriod=30 Sep 09 08:49:24.106482 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:24.106447 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" containerID="cri-o://857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff" gracePeriod=30 Sep 09 08:49:25.647669 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:25.647638 2577 generic.go:356] "Generic (PLEG): container finished" podID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerID="c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592" exitCode=0 Sep 09 08:49:25.647669 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:25.647665 2577 generic.go:356] "Generic (PLEG): container finished" podID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerID="b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed" exitCode=0 Sep 09 08:49:25.648193 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:25.647714 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerDied","Data":"c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592"} Sep 09 08:49:25.648193 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:25.647760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerDied","Data":"b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed"} Sep 09 08:49:25.650902 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:25.650879 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerID="857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff" exitCode=0 Sep 09 08:49:25.650902 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:25.650900 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerID="0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121" exitCode=0 Sep 09 08:49:25.651071 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:25.650926 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerDied","Data":"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff"} Sep 09 08:49:25.651071 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:25.650953 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerDied","Data":"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121"} Sep 09 08:49:30.707296 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:30.707244 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 09 08:49:30.913488 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:30.913430 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 09 08:49:30.913488 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:30.913463 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 09 08:49:31.374918 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:31.374874 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 09 08:49:40.707063 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:40.707021 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 09 08:49:40.913341 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:40.913292 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 09 08:49:40.913502 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:40.913310 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 09 08:49:41.374932 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:41.374892 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 09 08:49:50.707334 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:50.707289 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 09 08:49:50.707746 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:50.707409 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:49:50.913320 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:50.913276 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 09 08:49:50.913508 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:50.913276 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 09 08:49:50.913508 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:50.913460 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:49:50.913615 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:50.913532 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:49:51.374371 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:51.374326 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 09 08:49:51.374541 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:51.374476 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:49:54.227349 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.227322 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:49:54.313101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.313018 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-registry-tls\") pod \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " Sep 09 08:49:54.313101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.313057 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dg7j\" (UniqueName: \"kubernetes.io/projected/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-kube-api-access-7dg7j\") pod \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " Sep 09 08:49:54.313101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.313077 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-feast-data\") pod \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " Sep 09 08:49:54.313347 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.313114 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-online-tls\") pod \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " Sep 09 08:49:54.313347 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.313142 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls\") pod \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " Sep 09 08:49:54.313347 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.313170 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-offline-tls\") pod \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\" (UID: \"e9c895b4-ba55-49a5-97d6-5ae3402da5b1\") " Sep 09 08:49:54.313651 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.313628 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e9c895b4-ba55-49a5-97d6-5ae3402da5b1" (UID: "e9c895b4-ba55-49a5-97d6-5ae3402da5b1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 08:49:54.313779 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.313753 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-feast-data" (OuterVolumeSpecName: "feast-data") pod "e9c895b4-ba55-49a5-97d6-5ae3402da5b1" (UID: "e9c895b4-ba55-49a5-97d6-5ae3402da5b1"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:49:54.315457 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.315428 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-online-tls" (OuterVolumeSpecName: "online-tls") pod "e9c895b4-ba55-49a5-97d6-5ae3402da5b1" (UID: "e9c895b4-ba55-49a5-97d6-5ae3402da5b1"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:49:54.315556 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.315520 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e9c895b4-ba55-49a5-97d6-5ae3402da5b1" (UID: "e9c895b4-ba55-49a5-97d6-5ae3402da5b1"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:49:54.315668 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.315643 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e9c895b4-ba55-49a5-97d6-5ae3402da5b1" (UID: "e9c895b4-ba55-49a5-97d6-5ae3402da5b1"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:49:54.315768 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.315687 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-kube-api-access-7dg7j" (OuterVolumeSpecName: "kube-api-access-7dg7j") pod "e9c895b4-ba55-49a5-97d6-5ae3402da5b1" (UID: "e9c895b4-ba55-49a5-97d6-5ae3402da5b1"). InnerVolumeSpecName "kube-api-access-7dg7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:49:54.413789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.413755 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-online-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.413789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.413783 2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-ui-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.413789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.413792 2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-offline-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.413789 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.413799 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-registry-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.414073 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.413808 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dg7j\" (UniqueName: \"kubernetes.io/projected/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-kube-api-access-7dg7j\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.414073 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.413816 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c895b4-ba55-49a5-97d6-5ae3402da5b1-feast-data\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.742670 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.742646 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:49:54.773910 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.773880 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerID="50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f" exitCode=137 Sep 09 08:49:54.773910 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.773905 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerID="4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b" exitCode=137 Sep 09 08:49:54.774137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.773935 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" Sep 09 08:49:54.774137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.773957 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerDied","Data":"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f"} Sep 09 08:49:54.774137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.774019 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerDied","Data":"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b"} Sep 09 08:49:54.774137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.774035 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl" event={"ID":"1c4771d2-77d1-43e4-82c0-9a89fccebffc","Type":"ContainerDied","Data":"f883ae07e54af4c5b5f2050ed65c53bbf7022fbd5d19b1d4d1847cb7d85baf4c"} Sep 09 08:49:54.774137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.774057 2577 scope.go:117] "RemoveContainer" containerID="857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff" Sep 09 08:49:54.776958 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.776930 2577 generic.go:356] "Generic (PLEG): container finished" podID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerID="a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c" exitCode=137 Sep 09 08:49:54.777073 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.776989 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" Sep 09 08:49:54.777073 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.777014 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerDied","Data":"a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c"} Sep 09 08:49:54.777073 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.777052 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf" event={"ID":"e9c895b4-ba55-49a5-97d6-5ae3402da5b1","Type":"ContainerDied","Data":"dbcbc0e6565efd7ef2f65bcb6bbc6d754404d8b4e5a103034cda17fb28f4edb2"} Sep 09 08:49:54.786654 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.786633 2577 scope.go:117] "RemoveContainer" containerID="50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f" Sep 09 08:49:54.794840 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.794826 2577 scope.go:117] "RemoveContainer" containerID="0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121" Sep 09 08:49:54.797186 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.797169 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf"] Sep 09 08:49:54.800665 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.800645 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-75bb5589c6-kcqvf"] Sep 09 08:49:54.803384 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.803245 2577 scope.go:117] "RemoveContainer" containerID="4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b" Sep 09 08:49:54.810534 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.810512 2577 scope.go:117] "RemoveContainer" containerID="71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0" Sep 09 08:49:54.815662 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.815646 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-794l8\" (UniqueName: \"kubernetes.io/projected/1c4771d2-77d1-43e4-82c0-9a89fccebffc-kube-api-access-794l8\") pod \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " Sep 09 08:49:54.815732 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.815693 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls\") pod \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " Sep 09 08:49:54.815732 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.815711 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1c4771d2-77d1-43e4-82c0-9a89fccebffc-feast-data\") pod \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " Sep 09 08:49:54.815840 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.815736 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls\") pod \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " Sep 09 08:49:54.815840 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.815762 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-online-tls\") pod \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " Sep 09 08:49:54.815840 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.815796 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls\") pod \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\" (UID: \"1c4771d2-77d1-43e4-82c0-9a89fccebffc\") " Sep 09 08:49:54.816346 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.816226 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c4771d2-77d1-43e4-82c0-9a89fccebffc-feast-data" (OuterVolumeSpecName: "feast-data") pod "1c4771d2-77d1-43e4-82c0-9a89fccebffc" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:49:54.817816 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.817786 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-online-tls" (OuterVolumeSpecName: "online-tls") pod "1c4771d2-77d1-43e4-82c0-9a89fccebffc" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:49:54.817936 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.817918 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "1c4771d2-77d1-43e4-82c0-9a89fccebffc" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:49:54.818215 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.818181 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c4771d2-77d1-43e4-82c0-9a89fccebffc-kube-api-access-794l8" (OuterVolumeSpecName: "kube-api-access-794l8") pod "1c4771d2-77d1-43e4-82c0-9a89fccebffc" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc"). InnerVolumeSpecName "kube-api-access-794l8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:49:54.818279 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.818255 2577 scope.go:117] "RemoveContainer" containerID="99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60" Sep 09 08:49:54.818465 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.818447 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "1c4771d2-77d1-43e4-82c0-9a89fccebffc" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:49:54.818569 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.818552 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1c4771d2-77d1-43e4-82c0-9a89fccebffc" (UID: "1c4771d2-77d1-43e4-82c0-9a89fccebffc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:49:54.825432 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.825414 2577 scope.go:117] "RemoveContainer" containerID="857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff" Sep 09 08:49:54.825715 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.825683 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff\": container with ID starting with 857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff not found: ID does not exist" containerID="857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff" Sep 09 08:49:54.825764 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.825721 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff"} err="failed to get container status \"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff\": rpc error: code = NotFound desc = could not find container \"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff\": container with ID starting with 857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff not found: ID does not exist" Sep 09 08:49:54.825764 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.825736 2577 scope.go:117] "RemoveContainer" containerID="50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f" Sep 09 08:49:54.825993 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.825955 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f\": container with ID starting with 50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f not found: ID does not exist" containerID="50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f" Sep 09 08:49:54.826035 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.825989 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f"} err="failed to get container status \"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f\": rpc error: code = NotFound desc = could not find container \"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f\": container with ID starting with 50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f not found: ID does not exist" Sep 09 08:49:54.826035 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.826003 2577 scope.go:117] "RemoveContainer" containerID="0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121" Sep 09 08:49:54.826238 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.826222 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121\": container with ID starting with 0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121 not found: ID does not exist" containerID="0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121" Sep 09 08:49:54.826275 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.826241 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121"} err="failed to get container status \"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121\": rpc error: code = NotFound desc = could not find container \"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121\": container with ID starting with 0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121 not found: ID does not exist" Sep 09 08:49:54.826275 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.826252 2577 scope.go:117] "RemoveContainer" containerID="4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b" Sep 09 08:49:54.826482 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.826465 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b\": container with ID starting with 4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b not found: ID does not exist" containerID="4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b" Sep 09 08:49:54.826522 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.826488 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b"} err="failed to get container status \"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b\": rpc error: code = NotFound desc = could not find container \"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b\": container with ID starting with 4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b not found: ID does not exist" Sep 09 08:49:54.826522 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.826503 2577 scope.go:117] "RemoveContainer" containerID="71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0" Sep 09 08:49:54.826752 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.826738 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0\": container with ID starting with 71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0 not found: ID does not exist" containerID="71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0" Sep 09 08:49:54.826794 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.826754 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0"} err="failed to get container status \"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0\": rpc error: code = NotFound desc = could not find container \"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0\": container with ID starting with 71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0 not found: ID does not exist" Sep 09 08:49:54.826794 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.826767 2577 scope.go:117] "RemoveContainer" containerID="99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60" Sep 09 08:49:54.827131 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.827108 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60\": container with ID starting with 99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60 not found: ID does not exist" containerID="99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60" Sep 09 08:49:54.827190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.827136 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60"} err="failed to get container status \"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60\": rpc error: code = NotFound desc = could not find container \"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60\": container with ID starting with 99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60 not found: ID does not exist" Sep 09 08:49:54.827190 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.827151 2577 scope.go:117] "RemoveContainer" containerID="857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff" Sep 09 08:49:54.827409 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.827387 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff"} err="failed to get container status \"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff\": rpc error: code = NotFound desc = could not find container \"857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff\": container with ID starting with 857f32375f390dcd6e2781cfd07c142675bb0e57b5ce1e1f6376f3d0a79bc3ff not found: ID does not exist" Sep 09 08:49:54.827479 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.827411 2577 scope.go:117] "RemoveContainer" containerID="50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f" Sep 09 08:49:54.827639 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.827624 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f"} err="failed to get container status \"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f\": rpc error: code = NotFound desc = could not find container \"50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f\": container with ID starting with 50712cd57ea1e77f683368894b605857102aa9244af1b134d74d05531752170f not found: ID does not exist" Sep 09 08:49:54.827681 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.827640 2577 scope.go:117] "RemoveContainer" containerID="0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121" Sep 09 08:49:54.827847 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.827832 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121"} err="failed to get container status \"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121\": rpc error: code = NotFound desc = could not find container \"0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121\": container with ID starting with 0bfb36c4465a660f0c04eb436bc08f2ce2339f8d049ea71db52b5e9b6dbc2121 not found: ID does not exist" Sep 09 08:49:54.827886 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.827847 2577 scope.go:117] "RemoveContainer" containerID="4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b" Sep 09 08:49:54.828101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.828076 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b"} err="failed to get container status \"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b\": rpc error: code = NotFound desc = could not find container \"4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b\": container with ID starting with 4860e8cc98c38e2c36947632f7ea5abd2d6d4bd31cd5f67dc4a5d5718320e58b not found: ID does not exist" Sep 09 08:49:54.828101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.828100 2577 scope.go:117] "RemoveContainer" containerID="71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0" Sep 09 08:49:54.828305 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.828290 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0"} err="failed to get container status \"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0\": rpc error: code = NotFound desc = could not find container \"71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0\": container with ID starting with 71d811c9b502da3139a2c3fee3e5aaa53e99548225e0deb6b623ed605af7d2f0 not found: ID does not exist" Sep 09 08:49:54.828305 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.828305 2577 scope.go:117] "RemoveContainer" containerID="99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60" Sep 09 08:49:54.828498 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.828480 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60"} err="failed to get container status \"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60\": rpc error: code = NotFound desc = could not find container \"99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60\": container with ID starting with 99ed57fe3a8b6a203c5ed058639849011e932237f82cdbc317dd9ab719d44c60 not found: ID does not exist" Sep 09 08:49:54.828569 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.828500 2577 scope.go:117] "RemoveContainer" containerID="c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592" Sep 09 08:49:54.835369 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.835354 2577 scope.go:117] "RemoveContainer" containerID="a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c" Sep 09 08:49:54.842559 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.842541 2577 scope.go:117] "RemoveContainer" containerID="b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed" Sep 09 08:49:54.849254 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.849236 2577 scope.go:117] "RemoveContainer" containerID="b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61" Sep 09 08:49:54.856211 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.856187 2577 scope.go:117] "RemoveContainer" containerID="071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f" Sep 09 08:49:54.863055 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.863041 2577 scope.go:117] "RemoveContainer" containerID="c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592" Sep 09 08:49:54.863317 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.863300 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592\": container with ID starting with c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592 not found: ID does not exist" containerID="c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592" Sep 09 08:49:54.863399 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.863328 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592"} err="failed to get container status \"c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592\": rpc error: code = NotFound desc = could not find container \"c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592\": container with ID starting with c4aeed2c33383c93232f1c29ac232ab3e103842478baf8f34c2e156efdf5d592 not found: ID does not exist" Sep 09 08:49:54.863399 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.863351 2577 scope.go:117] "RemoveContainer" containerID="a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c" Sep 09 08:49:54.863617 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.863600 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c\": container with ID starting with a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c not found: ID does not exist" containerID="a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c" Sep 09 08:49:54.863653 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.863622 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c"} err="failed to get container status \"a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c\": rpc error: code = NotFound desc = could not find container \"a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c\": container with ID starting with a97cfc2da3b6d3629f61c7af179c95f3cd2c84de4b4f8bc7e3ad90d15a36680c not found: ID does not exist" Sep 09 08:49:54.863653 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.863638 2577 scope.go:117] "RemoveContainer" containerID="b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed" Sep 09 08:49:54.863876 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.863859 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed\": container with ID starting with b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed not found: ID does not exist" containerID="b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed" Sep 09 08:49:54.863964 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.863882 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed"} err="failed to get container status \"b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed\": rpc error: code = NotFound desc = could not find container \"b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed\": container with ID starting with b23073492065645945c18d13a0cd8bf4e32c402a01455c6d5b2e02230ceb19ed not found: ID does not exist" Sep 09 08:49:54.863964 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.863900 2577 scope.go:117] "RemoveContainer" containerID="b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61" Sep 09 08:49:54.864173 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.864157 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61\": container with ID starting with b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61 not found: ID does not exist" containerID="b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61" Sep 09 08:49:54.864215 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.864179 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61"} err="failed to get container status \"b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61\": rpc error: code = NotFound desc = could not find container \"b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61\": container with ID starting with b77d1cf1c9559ef7517bbcbf77111a8f92be38776e3f24d22df09ab6cb034a61 not found: ID does not exist" Sep 09 08:49:54.864215 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.864193 2577 scope.go:117] "RemoveContainer" containerID="071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f" Sep 09 08:49:54.864435 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:49:54.864419 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f\": container with ID starting with 071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f not found: ID does not exist" containerID="071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f" Sep 09 08:49:54.864486 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.864441 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f"} err="failed to get container status \"071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f\": rpc error: code = NotFound desc = could not find container \"071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f\": container with ID starting with 071d7ce8a3e4920ea23b696c2d2170740d8c92a73369d5f83880df353eb21d4f not found: ID does not exist" Sep 09 08:49:54.916601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.916522 2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-offline-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.916601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.916545 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1c4771d2-77d1-43e4-82c0-9a89fccebffc-feast-data\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.916601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.916553 2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-ui-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.916601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.916562 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-online-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.916601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.916571 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1c4771d2-77d1-43e4-82c0-9a89fccebffc-registry-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:54.916601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:54.916580 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-794l8\" (UniqueName: \"kubernetes.io/projected/1c4771d2-77d1-43e4-82c0-9a89fccebffc-kube-api-access-794l8\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:49:55.101600 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:55.101568 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl"] Sep 09 08:49:55.105499 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:55.105474 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c77f4b9-q9ggl"] Sep 09 08:49:56.633949 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:56.633918 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" path="/var/lib/kubelet/pods/1c4771d2-77d1-43e4-82c0-9a89fccebffc/volumes" Sep 09 08:49:56.634545 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:49:56.634530 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" path="/var/lib/kubelet/pods/e9c895b4-ba55-49a5-97d6-5ae3402da5b1/volumes" Sep 09 08:50:07.037335 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037287 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-tv2ch"] Sep 09 08:50:07.037815 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037798 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" Sep 09 08:50:07.037864 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037818 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" Sep 09 08:50:07.037864 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037842 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" Sep 09 08:50:07.037864 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037852 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037862 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="feast-init" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037871 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="feast-init" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037886 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037895 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037905 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="feast-init" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037913 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="feast-init" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037922 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="feast-apply" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037930 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="feast-apply" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037943 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="registry" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037951 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="registry" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037961 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="feast-apply" Sep 09 08:50:07.037991 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.037993 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="feast-apply" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038005 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="offline" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038013 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="offline" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038025 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="offline" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038033 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="offline" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038046 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038054 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038134 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="registry" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038143 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="offline" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038152 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="offline" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038161 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="online" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038170 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="ui" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038182 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c895b4-ba55-49a5-97d6-5ae3402da5b1" containerName="online" Sep 09 08:50:07.038379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.038194 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c4771d2-77d1-43e4-82c0-9a89fccebffc" containerName="ui" Sep 09 08:50:07.043436 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.043414 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:50:07.045926 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.045899 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 09 08:50:07.046162 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.046143 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 09 08:50:07.046233 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.046188 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-s7nlj\"" Sep 09 08:50:07.046292 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.046232 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 09 08:50:07.047339 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.047319 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tv2ch"] Sep 09 08:50:07.110948 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.110916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-postgresdata\") pod \"postgres-864846d4f5-tv2ch\" (UID: \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\") " pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:50:07.111116 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.111051 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx5cw\" (UniqueName: \"kubernetes.io/projected/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-kube-api-access-sx5cw\") pod \"postgres-864846d4f5-tv2ch\" (UID: \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\") " pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:50:07.211788 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.211754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx5cw\" (UniqueName: \"kubernetes.io/projected/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-kube-api-access-sx5cw\") pod \"postgres-864846d4f5-tv2ch\" (UID: \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\") " pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:50:07.211788 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.211794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-postgresdata\") pod \"postgres-864846d4f5-tv2ch\" (UID: \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\") " pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:50:07.212163 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.212147 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-postgresdata\") pod \"postgres-864846d4f5-tv2ch\" (UID: \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\") " pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:50:07.219896 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.219869 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx5cw\" (UniqueName: \"kubernetes.io/projected/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-kube-api-access-sx5cw\") pod \"postgres-864846d4f5-tv2ch\" (UID: \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\") " pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:50:07.363732 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.363630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:50:07.633182 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.633123 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tv2ch"] Sep 09 08:50:07.635471 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:50:07.635441 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee56c6c4_9ac5_4a39_830b_1b0dbd43f150.slice/crio-3b1f598f80886ea1ac8a510d92f0cccb7c8136c2cf5e878ec1e7b91e9a54d319 WatchSource:0}: Error finding container 3b1f598f80886ea1ac8a510d92f0cccb7c8136c2cf5e878ec1e7b91e9a54d319: Status 404 returned error can't find the container with id 3b1f598f80886ea1ac8a510d92f0cccb7c8136c2cf5e878ec1e7b91e9a54d319 Sep 09 08:50:07.637424 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.637409 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 08:50:07.834194 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:07.834161 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-tv2ch" event={"ID":"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150","Type":"ContainerStarted","Data":"3b1f598f80886ea1ac8a510d92f0cccb7c8136c2cf5e878ec1e7b91e9a54d319"} Sep 09 08:50:13.862738 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:13.862703 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-tv2ch" event={"ID":"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150","Type":"ContainerStarted","Data":"1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0"} Sep 09 08:50:13.878824 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:13.878762 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-tv2ch" podStartSLOduration=1.5330765610000001 podStartE2EDuration="6.878738404s" totalImagesPullingTime="5.345661843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:50:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:50:13.876059686 +0000 UTC m=+657.709423246" watchObservedRunningTime="2026-09-09 08:50:13.878738404 +0000 UTC m=+657.712101965" Sep 09 08:50:17.834128 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.834093 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2"] Sep 09 08:50:17.841129 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.841109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:17.843739 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.843721 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 09 08:50:17.843850 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.843829 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-n6r94\"" Sep 09 08:50:17.843901 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.843841 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 09 08:50:17.843957 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.843933 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 09 08:50:17.845016 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.844859 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 09 08:50:17.847704 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.847681 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2"] Sep 09 08:50:17.907958 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.907921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-online-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:17.908123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.907996 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-offline-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:17.908123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.908023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-registry-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:17.908123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.908050 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8r5w\" (UniqueName: \"kubernetes.io/projected/684260bb-e672-42cd-b176-b971571d950f-kube-api-access-b8r5w\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:17.908123 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:17.908067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/684260bb-e672-42cd-b176-b971571d950f-feast-data\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.008477 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.008447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8r5w\" (UniqueName: \"kubernetes.io/projected/684260bb-e672-42cd-b176-b971571d950f-kube-api-access-b8r5w\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.008623 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.008488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/684260bb-e672-42cd-b176-b971571d950f-feast-data\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.008623 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.008541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-online-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.008623 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.008581 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-offline-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.008623 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.008603 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-registry-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.008964 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.008943 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/684260bb-e672-42cd-b176-b971571d950f-feast-data\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.010835 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.010805 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-registry-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.010953 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.010868 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-online-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.011055 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.011000 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-offline-tls\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.017213 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.017192 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8r5w\" (UniqueName: \"kubernetes.io/projected/684260bb-e672-42cd-b176-b971571d950f-kube-api-access-b8r5w\") pod \"feast-credit-scoring-5849b45f4f-l86v2\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.152090 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.152003 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:18.430640 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.430616 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2"] Sep 09 08:50:18.433231 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:50:18.433201 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684260bb_e672_42cd_b176_b971571d950f.slice/crio-5d2723b86cd7e1093de69fab55a4d773ebcbc51a40ae2dcc9a8869131a6b6db9 WatchSource:0}: Error finding container 5d2723b86cd7e1093de69fab55a4d773ebcbc51a40ae2dcc9a8869131a6b6db9: Status 404 returned error can't find the container with id 5d2723b86cd7e1093de69fab55a4d773ebcbc51a40ae2dcc9a8869131a6b6db9 Sep 09 08:50:18.883694 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.883657 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerStarted","Data":"3ab49b48ee73587cbc757802a5501fd52cfb76e7f0868e2a1de64ca027d758b9"} Sep 09 08:50:18.884083 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:18.883721 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerStarted","Data":"5d2723b86cd7e1093de69fab55a4d773ebcbc51a40ae2dcc9a8869131a6b6db9"} Sep 09 08:50:20.893134 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:20.893102 2577 generic.go:356] "Generic (PLEG): container finished" podID="684260bb-e672-42cd-b176-b971571d950f" containerID="3ab49b48ee73587cbc757802a5501fd52cfb76e7f0868e2a1de64ca027d758b9" exitCode=0 Sep 09 08:50:20.893518 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:20.893139 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerDied","Data":"3ab49b48ee73587cbc757802a5501fd52cfb76e7f0868e2a1de64ca027d758b9"} Sep 09 08:50:21.899479 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:21.899447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerStarted","Data":"aa1f01ca7cb33dea1d4857488250972447aa37a5e63f0dff402cdc7e777d3d1b"} Sep 09 08:50:27.930200 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:27.930118 2577 generic.go:356] "Generic (PLEG): container finished" podID="684260bb-e672-42cd-b176-b971571d950f" containerID="aa1f01ca7cb33dea1d4857488250972447aa37a5e63f0dff402cdc7e777d3d1b" exitCode=0 Sep 09 08:50:27.930200 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:27.930190 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerDied","Data":"aa1f01ca7cb33dea1d4857488250972447aa37a5e63f0dff402cdc7e777d3d1b"} Sep 09 08:50:28.939294 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:28.939260 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerStarted","Data":"12d55d87c7c8dfe933cdc82546e9953cbbd3d8f66e97b10c745ff1bfd72a235a"} Sep 09 08:50:28.939771 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:28.939330 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerStarted","Data":"6a397964bf8e1c7d7d7d627a201d728481148041d30777298be2a161f62df015"} Sep 09 08:50:28.939771 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:28.939356 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerStarted","Data":"9429f3f64202c942a3f95da5c2f5b51f272afb055f6863c6d68aca942dbd8ae0"} Sep 09 08:50:28.965114 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:28.965049 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podStartSLOduration=11.965025476 podStartE2EDuration="11.965025476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:50:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:50:28.961429983 +0000 UTC m=+672.794793580" watchObservedRunningTime="2026-09-09 08:50:28.965025476 +0000 UTC m=+672.798389039" Sep 09 08:50:30.152191 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:30.152149 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:30.152831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:30.152795 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:30.152831 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:30.152836 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:30.153844 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:30.153810 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 09 08:50:30.153953 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:30.153831 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused" Sep 09 08:50:30.153953 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:30.153869 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 09 08:50:33.152287 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:33.152240 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 09 08:50:33.152770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:33.152435 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 09 08:50:33.152770 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:33.152435 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused" Sep 09 08:50:36.153374 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:36.153317 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 09 08:50:36.153848 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:36.153317 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused" Sep 09 08:50:36.153848 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:36.153337 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 09 08:50:38.152624 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:38.152574 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:38.153137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:38.152677 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:38.153137 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:38.152700 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:39.152894 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:39.152795 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 09 08:50:39.153724 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:39.153693 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:39.154610 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:39.154590 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:39.159085 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:39.159064 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:40.012196 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:40.012165 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:42.153736 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:42.153700 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:50:43.023278 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:50:43.023249 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:51:47.644544 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:47.644507 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2"] Sep 09 08:51:47.659720 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:47.659694 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tv2ch"] Sep 09 08:51:47.659889 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:47.659869 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-tv2ch" podUID="ee56c6c4-9ac5-4a39-830b-1b0dbd43f150" containerName="postgres" containerID="cri-o://1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0" gracePeriod=30 Sep 09 08:51:47.900683 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:47.900623 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:51:48.005986 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.005943 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-postgresdata\") pod \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\" (UID: \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\") " Sep 09 08:51:48.006145 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.006013 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx5cw\" (UniqueName: \"kubernetes.io/projected/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-kube-api-access-sx5cw\") pod \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\" (UID: \"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150\") " Sep 09 08:51:48.007822 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.007793 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-kube-api-access-sx5cw" (OuterVolumeSpecName: "kube-api-access-sx5cw") pod "ee56c6c4-9ac5-4a39-830b-1b0dbd43f150" (UID: "ee56c6c4-9ac5-4a39-830b-1b0dbd43f150"). InnerVolumeSpecName "kube-api-access-sx5cw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:51:48.034674 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.034650 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "ee56c6c4-9ac5-4a39-830b-1b0dbd43f150" (UID: "ee56c6c4-9ac5-4a39-830b-1b0dbd43f150"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:51:48.107330 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.107307 2577 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-postgresdata\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:51:48.107330 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.107328 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx5cw\" (UniqueName: \"kubernetes.io/projected/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150-kube-api-access-sx5cw\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:51:48.315839 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.315807 2577 generic.go:356] "Generic (PLEG): container finished" podID="ee56c6c4-9ac5-4a39-830b-1b0dbd43f150" containerID="1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0" exitCode=0 Sep 09 08:51:48.316030 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.315850 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-tv2ch" Sep 09 08:51:48.316030 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.315898 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-tv2ch" event={"ID":"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150","Type":"ContainerDied","Data":"1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0"} Sep 09 08:51:48.316030 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.315942 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-tv2ch" event={"ID":"ee56c6c4-9ac5-4a39-830b-1b0dbd43f150","Type":"ContainerDied","Data":"3b1f598f80886ea1ac8a510d92f0cccb7c8136c2cf5e878ec1e7b91e9a54d319"} Sep 09 08:51:48.316030 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.315997 2577 scope.go:117] "RemoveContainer" containerID="1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0" Sep 09 08:51:48.316237 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.316188 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="registry" containerID="cri-o://9429f3f64202c942a3f95da5c2f5b51f272afb055f6863c6d68aca942dbd8ae0" gracePeriod=30 Sep 09 08:51:48.316294 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.316263 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" containerID="cri-o://6a397964bf8e1c7d7d7d627a201d728481148041d30777298be2a161f62df015" gracePeriod=30 Sep 09 08:51:48.316349 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.316214 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="offline" containerID="cri-o://12d55d87c7c8dfe933cdc82546e9953cbbd3d8f66e97b10c745ff1bfd72a235a" gracePeriod=30 Sep 09 08:51:48.324643 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.324628 2577 scope.go:117] "RemoveContainer" containerID="1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0" Sep 09 08:51:48.324900 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:51:48.324880 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0\": container with ID starting with 1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0 not found: ID does not exist" containerID="1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0" Sep 09 08:51:48.325010 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.324906 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0"} err="failed to get container status \"1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0\": rpc error: code = NotFound desc = could not find container \"1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0\": container with ID starting with 1d719f96294276df460b848f6e100afb4820acf3032d2ee758ca3167e20d1dc0 not found: ID does not exist" Sep 09 08:51:48.351764 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.351739 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tv2ch"] Sep 09 08:51:48.355383 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.355364 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tv2ch"] Sep 09 08:51:48.635233 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:48.635156 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee56c6c4-9ac5-4a39-830b-1b0dbd43f150" path="/var/lib/kubelet/pods/ee56c6c4-9ac5-4a39-830b-1b0dbd43f150/volumes" Sep 09 08:51:49.322497 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:49.322465 2577 generic.go:356] "Generic (PLEG): container finished" podID="684260bb-e672-42cd-b176-b971571d950f" containerID="6a397964bf8e1c7d7d7d627a201d728481148041d30777298be2a161f62df015" exitCode=0 Sep 09 08:51:49.322889 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:49.322545 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerDied","Data":"6a397964bf8e1c7d7d7d627a201d728481148041d30777298be2a161f62df015"} Sep 09 08:51:50.009261 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:51:50.009219 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 09 08:52:00.009658 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:00.009611 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 09 08:52:10.009379 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:10.009335 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 09 08:52:10.009772 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:10.009463 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:52:18.452517 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.452476 2577 generic.go:356] "Generic (PLEG): container finished" podID="684260bb-e672-42cd-b176-b971571d950f" containerID="12d55d87c7c8dfe933cdc82546e9953cbbd3d8f66e97b10c745ff1bfd72a235a" exitCode=137 Sep 09 08:52:18.452517 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.452504 2577 generic.go:356] "Generic (PLEG): container finished" podID="684260bb-e672-42cd-b176-b971571d950f" containerID="9429f3f64202c942a3f95da5c2f5b51f272afb055f6863c6d68aca942dbd8ae0" exitCode=137 Sep 09 08:52:18.452919 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.452545 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerDied","Data":"12d55d87c7c8dfe933cdc82546e9953cbbd3d8f66e97b10c745ff1bfd72a235a"} Sep 09 08:52:18.452919 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.452619 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerDied","Data":"9429f3f64202c942a3f95da5c2f5b51f272afb055f6863c6d68aca942dbd8ae0"} Sep 09 08:52:18.966153 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.966127 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:52:18.975982 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.975950 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-online-tls\") pod \"684260bb-e672-42cd-b176-b971571d950f\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " Sep 09 08:52:18.976049 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.975998 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-registry-tls\") pod \"684260bb-e672-42cd-b176-b971571d950f\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " Sep 09 08:52:18.976049 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.976034 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-offline-tls\") pod \"684260bb-e672-42cd-b176-b971571d950f\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " Sep 09 08:52:18.976130 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.976072 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8r5w\" (UniqueName: \"kubernetes.io/projected/684260bb-e672-42cd-b176-b971571d950f-kube-api-access-b8r5w\") pod \"684260bb-e672-42cd-b176-b971571d950f\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " Sep 09 08:52:18.976130 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.976102 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/684260bb-e672-42cd-b176-b971571d950f-feast-data\") pod \"684260bb-e672-42cd-b176-b971571d950f\" (UID: \"684260bb-e672-42cd-b176-b971571d950f\") " Sep 09 08:52:18.978161 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.978132 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "684260bb-e672-42cd-b176-b971571d950f" (UID: "684260bb-e672-42cd-b176-b971571d950f"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:52:18.978315 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.978188 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/684260bb-e672-42cd-b176-b971571d950f-kube-api-access-b8r5w" (OuterVolumeSpecName: "kube-api-access-b8r5w") pod "684260bb-e672-42cd-b176-b971571d950f" (UID: "684260bb-e672-42cd-b176-b971571d950f"). InnerVolumeSpecName "kube-api-access-b8r5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:52:18.978415 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.978364 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-online-tls" (OuterVolumeSpecName: "online-tls") pod "684260bb-e672-42cd-b176-b971571d950f" (UID: "684260bb-e672-42cd-b176-b971571d950f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:52:18.978524 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.978416 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "684260bb-e672-42cd-b176-b971571d950f" (UID: "684260bb-e672-42cd-b176-b971571d950f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:52:18.984107 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:18.984073 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684260bb-e672-42cd-b176-b971571d950f-feast-data" (OuterVolumeSpecName: "feast-data") pod "684260bb-e672-42cd-b176-b971571d950f" (UID: "684260bb-e672-42cd-b176-b971571d950f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:52:19.076780 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.076746 2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-offline-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:52:19.076780 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.076773 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8r5w\" (UniqueName: \"kubernetes.io/projected/684260bb-e672-42cd-b176-b971571d950f-kube-api-access-b8r5w\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:52:19.076780 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.076782 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/684260bb-e672-42cd-b176-b971571d950f-feast-data\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:52:19.077039 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.076812 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-online-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:52:19.077039 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.076820 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/684260bb-e672-42cd-b176-b971571d950f-registry-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:52:19.459989 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.459934 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" Sep 09 08:52:19.459989 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.459944 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2" event={"ID":"684260bb-e672-42cd-b176-b971571d950f","Type":"ContainerDied","Data":"5d2723b86cd7e1093de69fab55a4d773ebcbc51a40ae2dcc9a8869131a6b6db9"} Sep 09 08:52:19.460517 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.460036 2577 scope.go:117] "RemoveContainer" containerID="12d55d87c7c8dfe933cdc82546e9953cbbd3d8f66e97b10c745ff1bfd72a235a" Sep 09 08:52:19.469576 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.469558 2577 scope.go:117] "RemoveContainer" containerID="6a397964bf8e1c7d7d7d627a201d728481148041d30777298be2a161f62df015" Sep 09 08:52:19.477340 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.477319 2577 scope.go:117] "RemoveContainer" containerID="9429f3f64202c942a3f95da5c2f5b51f272afb055f6863c6d68aca942dbd8ae0" Sep 09 08:52:19.482638 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.482615 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2"] Sep 09 08:52:19.485516 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.485499 2577 scope.go:117] "RemoveContainer" containerID="aa1f01ca7cb33dea1d4857488250972447aa37a5e63f0dff402cdc7e777d3d1b" Sep 09 08:52:19.485778 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.485757 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-5849b45f4f-l86v2"] Sep 09 08:52:19.492471 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:19.492452 2577 scope.go:117] "RemoveContainer" containerID="3ab49b48ee73587cbc757802a5501fd52cfb76e7f0868e2a1de64ca027d758b9" Sep 09 08:52:20.633682 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:52:20.633640 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="684260bb-e672-42cd-b176-b971571d950f" path="/var/lib/kubelet/pods/684260bb-e672-42cd-b176-b971571d950f/volumes" Sep 09 08:54:14.820302 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820228 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-lzbs8"] Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820656 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820673 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820695 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="feast-init" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820706 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="feast-init" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820727 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee56c6c4-9ac5-4a39-830b-1b0dbd43f150" containerName="postgres" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820736 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee56c6c4-9ac5-4a39-830b-1b0dbd43f150" containerName="postgres" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820750 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="feast-apply" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820758 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="feast-apply" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820767 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="registry" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820775 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="registry" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820789 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="offline" Sep 09 08:54:14.820813 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820797 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="offline" Sep 09 08:54:14.821343 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820866 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="registry" Sep 09 08:54:14.821343 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820876 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="online" Sep 09 08:54:14.821343 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820884 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee56c6c4-9ac5-4a39-830b-1b0dbd43f150" containerName="postgres" Sep 09 08:54:14.821343 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.820894 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="684260bb-e672-42cd-b176-b971571d950f" containerName="offline" Sep 09 08:54:14.823756 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.823736 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" Sep 09 08:54:14.826334 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.826317 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 09 08:54:14.827573 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.827557 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-9fm6z\"" Sep 09 08:54:14.827638 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.827578 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 09 08:54:14.839068 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.838818 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-lzbs8"] Sep 09 08:54:14.955241 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:14.955210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxkt4\" (UniqueName: \"kubernetes.io/projected/7e7f29b2-cc1c-475e-be20-e92f9437777f-kube-api-access-cxkt4\") pod \"redis-69bbd798b6-lzbs8\" (UID: \"7e7f29b2-cc1c-475e-be20-e92f9437777f\") " pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" Sep 09 08:54:15.056338 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:15.056300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxkt4\" (UniqueName: \"kubernetes.io/projected/7e7f29b2-cc1c-475e-be20-e92f9437777f-kube-api-access-cxkt4\") pod \"redis-69bbd798b6-lzbs8\" (UID: \"7e7f29b2-cc1c-475e-be20-e92f9437777f\") " pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" Sep 09 08:54:15.064745 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:15.064716 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxkt4\" (UniqueName: \"kubernetes.io/projected/7e7f29b2-cc1c-475e-be20-e92f9437777f-kube-api-access-cxkt4\") pod \"redis-69bbd798b6-lzbs8\" (UID: \"7e7f29b2-cc1c-475e-be20-e92f9437777f\") " pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" Sep 09 08:54:15.167514 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:15.167444 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" Sep 09 08:54:15.434091 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:15.434058 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-lzbs8"] Sep 09 08:54:15.435800 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:54:15.435776 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e7f29b2_cc1c_475e_be20_e92f9437777f.slice/crio-1dc8bfc3d6d37baa3be913a005cb35c3bcc4b39bb7a4cd11e23bbb4892d43473 WatchSource:0}: Error finding container 1dc8bfc3d6d37baa3be913a005cb35c3bcc4b39bb7a4cd11e23bbb4892d43473: Status 404 returned error can't find the container with id 1dc8bfc3d6d37baa3be913a005cb35c3bcc4b39bb7a4cd11e23bbb4892d43473 Sep 09 08:54:15.922503 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:15.922469 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" event={"ID":"7e7f29b2-cc1c-475e-be20-e92f9437777f","Type":"ContainerStarted","Data":"1dc8bfc3d6d37baa3be913a005cb35c3bcc4b39bb7a4cd11e23bbb4892d43473"} Sep 09 08:54:16.928164 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:16.928125 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" event={"ID":"7e7f29b2-cc1c-475e-be20-e92f9437777f","Type":"ContainerStarted","Data":"b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034"} Sep 09 08:54:16.943092 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:16.943046 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" podStartSLOduration=1.655075927 podStartE2EDuration="2.943032083s" totalImagesPullingTime="1.287956156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:54:16.941934175 +0000 UTC m=+900.775297734" watchObservedRunningTime="2026-09-09 08:54:16.943032083 +0000 UTC m=+900.776395641" Sep 09 08:54:20.235707 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.235674 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-d95gl"] Sep 09 08:54:20.239220 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.239200 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:54:20.242005 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.241793 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 09 08:54:20.244365 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.244339 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-d95gl"] Sep 09 08:54:20.299040 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.299010 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw4tg\" (UniqueName: \"kubernetes.io/projected/bb9902ea-bc33-496a-bab8-676aa27027ce-kube-api-access-sw4tg\") pod \"postgres-864846d4f5-d95gl\" (UID: \"bb9902ea-bc33-496a-bab8-676aa27027ce\") " pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:54:20.299194 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.299049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bb9902ea-bc33-496a-bab8-676aa27027ce-postgresdata\") pod \"postgres-864846d4f5-d95gl\" (UID: \"bb9902ea-bc33-496a-bab8-676aa27027ce\") " pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:54:20.400121 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.400089 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw4tg\" (UniqueName: \"kubernetes.io/projected/bb9902ea-bc33-496a-bab8-676aa27027ce-kube-api-access-sw4tg\") pod \"postgres-864846d4f5-d95gl\" (UID: \"bb9902ea-bc33-496a-bab8-676aa27027ce\") " pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:54:20.400121 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.400126 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bb9902ea-bc33-496a-bab8-676aa27027ce-postgresdata\") pod \"postgres-864846d4f5-d95gl\" (UID: \"bb9902ea-bc33-496a-bab8-676aa27027ce\") " pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:54:20.400451 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.400436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bb9902ea-bc33-496a-bab8-676aa27027ce-postgresdata\") pod \"postgres-864846d4f5-d95gl\" (UID: \"bb9902ea-bc33-496a-bab8-676aa27027ce\") " pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:54:20.408630 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.408600 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw4tg\" (UniqueName: \"kubernetes.io/projected/bb9902ea-bc33-496a-bab8-676aa27027ce-kube-api-access-sw4tg\") pod \"postgres-864846d4f5-d95gl\" (UID: \"bb9902ea-bc33-496a-bab8-676aa27027ce\") " pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:54:20.550340 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.550314 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:54:20.816630 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.816555 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-d95gl"] Sep 09 08:54:20.819191 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:54:20.819165 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb9902ea_bc33_496a_bab8_676aa27027ce.slice/crio-246f12e946c9262ae2138236fb92f2424c1b3d4f15bb501c3ed682b1e8bcee7e WatchSource:0}: Error finding container 246f12e946c9262ae2138236fb92f2424c1b3d4f15bb501c3ed682b1e8bcee7e: Status 404 returned error can't find the container with id 246f12e946c9262ae2138236fb92f2424c1b3d4f15bb501c3ed682b1e8bcee7e Sep 09 08:54:20.946159 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.946125 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" event={"ID":"bb9902ea-bc33-496a-bab8-676aa27027ce","Type":"ContainerStarted","Data":"57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff"} Sep 09 08:54:20.946304 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.946168 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" event={"ID":"bb9902ea-bc33-496a-bab8-676aa27027ce","Type":"ContainerStarted","Data":"246f12e946c9262ae2138236fb92f2424c1b3d4f15bb501c3ed682b1e8bcee7e"} Sep 09 08:54:20.961868 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:20.961816 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" podStartSLOduration=0.961799444 podStartE2EDuration="961.799444ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:54:20.961635147 +0000 UTC m=+904.794998706" watchObservedRunningTime="2026-09-09 08:54:20.961799444 +0000 UTC m=+904.795163004" Sep 09 08:54:25.765872 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.765837 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq"] Sep 09 08:54:25.773903 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.773882 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.776571 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.776548 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 09 08:54:25.776685 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.776548 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 09 08:54:25.776685 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.776591 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-x6k2j\"" Sep 09 08:54:25.776685 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.776554 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 09 08:54:25.779790 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.779771 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq"] Sep 09 08:54:25.849466 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.849434 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-registry-tls\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.849600 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.849485 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5ba562-436b-491e-b635-bb1e15a58a0d-feast-data\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.849600 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.849540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-online-tls\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.849600 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.849562 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqbwz\" (UniqueName: \"kubernetes.io/projected/6b5ba562-436b-491e-b635-bb1e15a58a0d-kube-api-access-wqbwz\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.950865 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.950835 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5ba562-436b-491e-b635-bb1e15a58a0d-feast-data\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.951054 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.950879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-online-tls\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.951054 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.950900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqbwz\" (UniqueName: \"kubernetes.io/projected/6b5ba562-436b-491e-b635-bb1e15a58a0d-kube-api-access-wqbwz\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.951054 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.950941 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-registry-tls\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.951317 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.951295 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5ba562-436b-491e-b635-bb1e15a58a0d-feast-data\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.953378 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.953358 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-online-tls\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.953513 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.953397 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-registry-tls\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:25.958595 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:25.958571 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqbwz\" (UniqueName: \"kubernetes.io/projected/6b5ba562-436b-491e-b635-bb1e15a58a0d-kube-api-access-wqbwz\") pod \"feast-credit-scoring-86f966697-2vgtq\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:26.084824 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:26.084742 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:26.358223 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:26.358200 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq"] Sep 09 08:54:26.360212 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:54:26.360187 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5ba562_436b_491e_b635_bb1e15a58a0d.slice/crio-1af5b62ee9805a891828dfd186ec363e08dd2e5ecae00f9e64d32b0818eb2e00 WatchSource:0}: Error finding container 1af5b62ee9805a891828dfd186ec363e08dd2e5ecae00f9e64d32b0818eb2e00: Status 404 returned error can't find the container with id 1af5b62ee9805a891828dfd186ec363e08dd2e5ecae00f9e64d32b0818eb2e00 Sep 09 08:54:26.974412 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:26.974380 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerStarted","Data":"640ded2747c1ee5e6f2e1287b72e936404f8c19ab579e155b550fd129f8968cf"} Sep 09 08:54:26.974817 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:26.974430 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerStarted","Data":"1af5b62ee9805a891828dfd186ec363e08dd2e5ecae00f9e64d32b0818eb2e00"} Sep 09 08:54:27.980112 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:27.980082 2577 generic.go:356] "Generic (PLEG): container finished" podID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerID="640ded2747c1ee5e6f2e1287b72e936404f8c19ab579e155b550fd129f8968cf" exitCode=0 Sep 09 08:54:27.980533 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:27.980118 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerDied","Data":"640ded2747c1ee5e6f2e1287b72e936404f8c19ab579e155b550fd129f8968cf"} Sep 09 08:54:28.987596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:28.987560 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerStarted","Data":"638df6566311c9204df53c1b6d098f508ebbb8f92cb3152d52720a765ae163c5"} Sep 09 08:54:35.013784 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:35.013752 2577 generic.go:356] "Generic (PLEG): container finished" podID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerID="638df6566311c9204df53c1b6d098f508ebbb8f92cb3152d52720a765ae163c5" exitCode=0 Sep 09 08:54:35.014216 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:35.013791 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerDied","Data":"638df6566311c9204df53c1b6d098f508ebbb8f92cb3152d52720a765ae163c5"} Sep 09 08:54:36.021411 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:36.021376 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerStarted","Data":"53b6f2133ed33c525572254d88658168cbe8e3153caaa4ff8de40c0f470456f2"} Sep 09 08:54:36.021884 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:36.021439 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerStarted","Data":"62e0ea325d6b33d1fbbc6f640f38a92eadc0529df591dbc7d52684a79bb6df4a"} Sep 09 08:54:36.043206 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:36.043158 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podStartSLOduration=11.04314113 podStartE2EDuration="11.04314113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:54:36.040896214 +0000 UTC m=+919.874259774" watchObservedRunningTime="2026-09-09 08:54:36.04314113 +0000 UTC m=+919.876504689" Sep 09 08:54:36.085517 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:36.085476 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:36.085700 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:36.085570 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:38.085712 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:38.085670 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:38.086213 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:38.085755 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:38.087103 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:38.087074 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 09 08:54:38.087234 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:38.087118 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 09 08:54:41.085990 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:41.085933 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 09 08:54:41.086470 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:41.085967 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 09 08:54:44.085147 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:44.085101 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 09 08:54:44.091051 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:44.091022 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:44.094311 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:44.094287 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:47.086240 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:47.086213 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:47.086913 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:47.086895 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:54:51.558309 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.558274 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv"] Sep 09 08:54:51.562088 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.562068 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.564700 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.564668 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 09 08:54:51.564818 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.564699 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 09 08:54:51.564818 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.564708 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 09 08:54:51.564818 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.564699 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-bs9dp\"" Sep 09 08:54:51.572224 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.572205 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv"] Sep 09 08:54:51.583523 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.583500 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.583621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.583538 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/60f30d66-5861-4cb8-83fd-f2844af8af40-feast-data\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.583621 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.583563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.583706 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.583669 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpxgx\" (UniqueName: \"kubernetes.io/projected/60f30d66-5861-4cb8-83fd-f2844af8af40-kube-api-access-dpxgx\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.684995 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.684935 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpxgx\" (UniqueName: \"kubernetes.io/projected/60f30d66-5861-4cb8-83fd-f2844af8af40-kube-api-access-dpxgx\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.685201 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.685039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.685201 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.685066 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/60f30d66-5861-4cb8-83fd-f2844af8af40-feast-data\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.685201 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.685085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.685201 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:54:51.685174 2577 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found Sep 09 08:54:51.685426 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:54:51.685228 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls podName:60f30d66-5861-4cb8-83fd-f2844af8af40 nodeName:}" failed. No retries permitted until 2026-09-09 08:54:52.185213505 +0000 UTC m=+936.018577041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls") pod "feast-driver-ranking-57dbd78c8b-69sxv" (UID: "60f30d66-5861-4cb8-83fd-f2844af8af40") : secret "feast-driver-ranking-registry-tls" not found Sep 09 08:54:51.685426 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:54:51.685176 2577 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 09 08:54:51.685426 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:54:51.685327 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls podName:60f30d66-5861-4cb8-83fd-f2844af8af40 nodeName:}" failed. No retries permitted until 2026-09-09 08:54:52.185313301 +0000 UTC m=+936.018676868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls") pod "feast-driver-ranking-57dbd78c8b-69sxv" (UID: "60f30d66-5861-4cb8-83fd-f2844af8af40") : secret "feast-driver-ranking-online-tls" not found Sep 09 08:54:51.685587 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.685510 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/60f30d66-5861-4cb8-83fd-f2844af8af40-feast-data\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:51.694417 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:51.694387 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpxgx\" (UniqueName: \"kubernetes.io/projected/60f30d66-5861-4cb8-83fd-f2844af8af40-kube-api-access-dpxgx\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:52.188741 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:52.188701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:52.188929 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:52.188794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:52.188929 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:54:52.188893 2577 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 09 08:54:52.189038 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:54:52.188953 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls podName:60f30d66-5861-4cb8-83fd-f2844af8af40 nodeName:}" failed. No retries permitted until 2026-09-09 08:54:53.188939636 +0000 UTC m=+937.022303172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls") pod "feast-driver-ranking-57dbd78c8b-69sxv" (UID: "60f30d66-5861-4cb8-83fd-f2844af8af40") : secret "feast-driver-ranking-online-tls" not found Sep 09 08:54:52.191154 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:52.191124 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:53.197114 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:53.197079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:53.199500 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:53.199475 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls\") pod \"feast-driver-ranking-57dbd78c8b-69sxv\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:53.371801 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:53.371754 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:54:53.643392 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:53.643362 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv"] Sep 09 08:54:53.644457 ip-10-0-133-59 kubenswrapper[2577]: W0909 08:54:53.644428 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f30d66_5861_4cb8_83fd_f2844af8af40.slice/crio-fca07c2f71d2f02fa7b46be910b8e3f658fa8275574df3ea815a3038db0ef365 WatchSource:0}: Error finding container fca07c2f71d2f02fa7b46be910b8e3f658fa8275574df3ea815a3038db0ef365: Status 404 returned error can't find the container with id fca07c2f71d2f02fa7b46be910b8e3f658fa8275574df3ea815a3038db0ef365 Sep 09 08:54:54.111763 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:54.111728 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerStarted","Data":"f01923cd02c624cc395e7904b1079c9845cf5a79a0ed0a30e254567e8fb823a6"} Sep 09 08:54:54.111935 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:54.111786 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerStarted","Data":"fca07c2f71d2f02fa7b46be910b8e3f658fa8275574df3ea815a3038db0ef365"} Sep 09 08:54:59.137715 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:59.137679 2577 generic.go:356] "Generic (PLEG): container finished" podID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerID="f01923cd02c624cc395e7904b1079c9845cf5a79a0ed0a30e254567e8fb823a6" exitCode=0 Sep 09 08:54:59.138102 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:54:59.137751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerDied","Data":"f01923cd02c624cc395e7904b1079c9845cf5a79a0ed0a30e254567e8fb823a6"} Sep 09 08:55:00.144174 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:00.144141 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerStarted","Data":"be450ef953e47db9288152ed997a3c2a7982f7338275aa9a57fcd74b9f3e6290"} Sep 09 08:55:06.173626 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:06.173593 2577 generic.go:356] "Generic (PLEG): container finished" podID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerID="be450ef953e47db9288152ed997a3c2a7982f7338275aa9a57fcd74b9f3e6290" exitCode=0 Sep 09 08:55:06.173998 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:06.173640 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerDied","Data":"be450ef953e47db9288152ed997a3c2a7982f7338275aa9a57fcd74b9f3e6290"} Sep 09 08:55:07.182652 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:07.182620 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerStarted","Data":"ab7d84c86f03fc900ce9cd94a287644abe69eda65c79d77c695cebaeeb248ae5"} Sep 09 08:55:07.183073 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:07.182673 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerStarted","Data":"c3027137257c12149921365c20511cdd791f801cd52fc9190f1d348330f20ea1"} Sep 09 08:55:07.204654 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:07.204602 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podStartSLOduration=16.204586144 podStartE2EDuration="16.204586144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 08:54:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 08:55:07.203288709 +0000 UTC m=+951.036652292" watchObservedRunningTime="2026-09-09 08:55:07.204586144 +0000 UTC m=+951.037949703" Sep 09 08:55:08.372541 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:08.372490 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:55:08.372945 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:08.372568 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:55:08.374153 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:08.374122 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 09 08:55:08.374275 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:08.374161 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 09 08:55:11.373089 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:11.373031 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 09 08:55:11.373609 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:11.373114 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 09 08:55:13.372923 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:13.372877 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:55:13.373427 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:13.372954 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:55:14.372507 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:14.372454 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 09 08:55:14.378524 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:14.378493 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:55:14.382052 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:14.382019 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:55:17.373077 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:17.373037 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 09 08:55:20.373238 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:20.373204 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:55:20.374032 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:55:20.374012 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:56:40.275549 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:40.275513 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq"] Sep 09 08:56:40.276346 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:40.276309 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="registry" containerID="cri-o://62e0ea325d6b33d1fbbc6f640f38a92eadc0529df591dbc7d52684a79bb6df4a" gracePeriod=30 Sep 09 08:56:40.276504 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:40.276323 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" containerID="cri-o://53b6f2133ed33c525572254d88658168cbe8e3153caaa4ff8de40c0f470456f2" gracePeriod=30 Sep 09 08:56:40.278227 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:40.278193 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv"] Sep 09 08:56:40.278490 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:40.278434 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" containerID="cri-o://c3027137257c12149921365c20511cdd791f801cd52fc9190f1d348330f20ea1" gracePeriod=30 Sep 09 08:56:40.278570 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:40.278480 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" containerID="cri-o://ab7d84c86f03fc900ce9cd94a287644abe69eda65c79d77c695cebaeeb248ae5" gracePeriod=30 Sep 09 08:56:41.626202 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:41.626169 2577 generic.go:356] "Generic (PLEG): container finished" podID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerID="ab7d84c86f03fc900ce9cd94a287644abe69eda65c79d77c695cebaeeb248ae5" exitCode=0 Sep 09 08:56:41.626628 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:41.626247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerDied","Data":"ab7d84c86f03fc900ce9cd94a287644abe69eda65c79d77c695cebaeeb248ae5"} Sep 09 08:56:41.628600 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:41.628579 2577 generic.go:356] "Generic (PLEG): container finished" podID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerID="53b6f2133ed33c525572254d88658168cbe8e3153caaa4ff8de40c0f470456f2" exitCode=0 Sep 09 08:56:41.628714 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:41.628618 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerDied","Data":"53b6f2133ed33c525572254d88658168cbe8e3153caaa4ff8de40c0f470456f2"} Sep 09 08:56:42.738858 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:42.738820 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-d95gl"] Sep 09 08:56:42.739267 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:42.739010 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" podUID="bb9902ea-bc33-496a-bab8-676aa27027ce" containerName="postgres" containerID="cri-o://57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff" gracePeriod=30 Sep 09 08:56:42.745448 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:42.745419 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-lzbs8"] Sep 09 08:56:42.745861 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:42.745831 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" podUID="7e7f29b2-cc1c-475e-be20-e92f9437777f" containerName="redis" containerID="cri-o://b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034" gracePeriod=30 Sep 09 08:56:42.991845 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:42.991788 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:56:43.025336 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.025302 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" Sep 09 08:56:43.079146 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.079117 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxkt4\" (UniqueName: \"kubernetes.io/projected/7e7f29b2-cc1c-475e-be20-e92f9437777f-kube-api-access-cxkt4\") pod \"7e7f29b2-cc1c-475e-be20-e92f9437777f\" (UID: \"7e7f29b2-cc1c-475e-be20-e92f9437777f\") " Sep 09 08:56:43.079341 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.079159 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw4tg\" (UniqueName: \"kubernetes.io/projected/bb9902ea-bc33-496a-bab8-676aa27027ce-kube-api-access-sw4tg\") pod \"bb9902ea-bc33-496a-bab8-676aa27027ce\" (UID: \"bb9902ea-bc33-496a-bab8-676aa27027ce\") " Sep 09 08:56:43.079341 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.079182 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bb9902ea-bc33-496a-bab8-676aa27027ce-postgresdata\") pod \"bb9902ea-bc33-496a-bab8-676aa27027ce\" (UID: \"bb9902ea-bc33-496a-bab8-676aa27027ce\") " Sep 09 08:56:43.081463 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.081423 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e7f29b2-cc1c-475e-be20-e92f9437777f-kube-api-access-cxkt4" (OuterVolumeSpecName: "kube-api-access-cxkt4") pod "7e7f29b2-cc1c-475e-be20-e92f9437777f" (UID: "7e7f29b2-cc1c-475e-be20-e92f9437777f"). InnerVolumeSpecName "kube-api-access-cxkt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:56:43.081574 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.081538 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb9902ea-bc33-496a-bab8-676aa27027ce-kube-api-access-sw4tg" (OuterVolumeSpecName: "kube-api-access-sw4tg") pod "bb9902ea-bc33-496a-bab8-676aa27027ce" (UID: "bb9902ea-bc33-496a-bab8-676aa27027ce"). InnerVolumeSpecName "kube-api-access-sw4tg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:56:43.109725 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.109690 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb9902ea-bc33-496a-bab8-676aa27027ce-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "bb9902ea-bc33-496a-bab8-676aa27027ce" (UID: "bb9902ea-bc33-496a-bab8-676aa27027ce"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:56:43.180650 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.180617 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxkt4\" (UniqueName: \"kubernetes.io/projected/7e7f29b2-cc1c-475e-be20-e92f9437777f-kube-api-access-cxkt4\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:56:43.180650 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.180648 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw4tg\" (UniqueName: \"kubernetes.io/projected/bb9902ea-bc33-496a-bab8-676aa27027ce-kube-api-access-sw4tg\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:56:43.180840 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.180663 2577 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bb9902ea-bc33-496a-bab8-676aa27027ce-postgresdata\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:56:43.644396 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.644360 2577 generic.go:356] "Generic (PLEG): container finished" podID="bb9902ea-bc33-496a-bab8-676aa27027ce" containerID="57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff" exitCode=0 Sep 09 08:56:43.644596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.644411 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" Sep 09 08:56:43.644596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.644446 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" event={"ID":"bb9902ea-bc33-496a-bab8-676aa27027ce","Type":"ContainerDied","Data":"57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff"} Sep 09 08:56:43.644596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.644492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-d95gl" event={"ID":"bb9902ea-bc33-496a-bab8-676aa27027ce","Type":"ContainerDied","Data":"246f12e946c9262ae2138236fb92f2424c1b3d4f15bb501c3ed682b1e8bcee7e"} Sep 09 08:56:43.644596 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.644515 2577 scope.go:117] "RemoveContainer" containerID="57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff" Sep 09 08:56:43.645810 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.645787 2577 generic.go:356] "Generic (PLEG): container finished" podID="7e7f29b2-cc1c-475e-be20-e92f9437777f" containerID="b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034" exitCode=0 Sep 09 08:56:43.645940 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.645833 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" Sep 09 08:56:43.645940 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.645854 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" event={"ID":"7e7f29b2-cc1c-475e-be20-e92f9437777f","Type":"ContainerDied","Data":"b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034"} Sep 09 08:56:43.645940 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.645882 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-lzbs8" event={"ID":"7e7f29b2-cc1c-475e-be20-e92f9437777f","Type":"ContainerDied","Data":"1dc8bfc3d6d37baa3be913a005cb35c3bcc4b39bb7a4cd11e23bbb4892d43473"} Sep 09 08:56:43.654363 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.654348 2577 scope.go:117] "RemoveContainer" containerID="57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff" Sep 09 08:56:43.654622 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:56:43.654603 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff\": container with ID starting with 57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff not found: ID does not exist" containerID="57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff" Sep 09 08:56:43.654665 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.654630 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff"} err="failed to get container status \"57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff\": rpc error: code = NotFound desc = could not find container \"57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff\": container with ID starting with 57380432e0f00d744a8d72cf8e1eb357bbef94b7e0eb458159df71ae4d8b25ff not found: ID does not exist" Sep 09 08:56:43.654665 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.654645 2577 scope.go:117] "RemoveContainer" containerID="b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034" Sep 09 08:56:43.662187 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.662170 2577 scope.go:117] "RemoveContainer" containerID="b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034" Sep 09 08:56:43.662435 ip-10-0-133-59 kubenswrapper[2577]: E0909 08:56:43.662416 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034\": container with ID starting with b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034 not found: ID does not exist" containerID="b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034" Sep 09 08:56:43.662500 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.662441 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034"} err="failed to get container status \"b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034\": rpc error: code = NotFound desc = could not find container \"b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034\": container with ID starting with b300c50849a153a40759b7ea38c7af12e1b631c2a916a0a53061e797951a1034 not found: ID does not exist" Sep 09 08:56:43.668607 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.668583 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-d95gl"] Sep 09 08:56:43.674458 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.674439 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-d95gl"] Sep 09 08:56:43.685218 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.685200 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-lzbs8"] Sep 09 08:56:43.688276 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:43.688257 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-lzbs8"] Sep 09 08:56:44.091893 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:44.091857 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 09 08:56:44.379841 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:44.379746 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 09 08:56:44.634229 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:44.634154 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e7f29b2-cc1c-475e-be20-e92f9437777f" path="/var/lib/kubelet/pods/7e7f29b2-cc1c-475e-be20-e92f9437777f/volumes" Sep 09 08:56:44.634493 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:44.634481 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb9902ea-bc33-496a-bab8-676aa27027ce" path="/var/lib/kubelet/pods/bb9902ea-bc33-496a-bab8-676aa27027ce/volumes" Sep 09 08:56:54.092455 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:54.092409 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 09 08:56:54.379323 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:56:54.379230 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 09 08:57:04.092451 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:04.092411 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 09 08:57:04.092822 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:04.092522 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:57:04.379149 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:04.379053 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 09 08:57:04.379297 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:04.379190 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:57:10.374266 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:10.374225 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 09 08:57:10.768586 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:10.768553 2577 generic.go:356] "Generic (PLEG): container finished" podID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerID="62e0ea325d6b33d1fbbc6f640f38a92eadc0529df591dbc7d52684a79bb6df4a" exitCode=137 Sep 09 08:57:10.768771 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:10.768623 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerDied","Data":"62e0ea325d6b33d1fbbc6f640f38a92eadc0529df591dbc7d52684a79bb6df4a"} Sep 09 08:57:10.771059 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:10.771035 2577 generic.go:356] "Generic (PLEG): container finished" podID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerID="c3027137257c12149921365c20511cdd791f801cd52fc9190f1d348330f20ea1" exitCode=137 Sep 09 08:57:10.771176 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:10.771082 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerDied","Data":"c3027137257c12149921365c20511cdd791f801cd52fc9190f1d348330f20ea1"} Sep 09 08:57:10.957827 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:10.957800 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:57:10.960344 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:10.960324 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:57:11.031833 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.031756 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5ba562-436b-491e-b635-bb1e15a58a0d-feast-data\") pod \"6b5ba562-436b-491e-b635-bb1e15a58a0d\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " Sep 09 08:57:11.031833 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.031816 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqbwz\" (UniqueName: \"kubernetes.io/projected/6b5ba562-436b-491e-b635-bb1e15a58a0d-kube-api-access-wqbwz\") pod \"6b5ba562-436b-491e-b635-bb1e15a58a0d\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " Sep 09 08:57:11.032101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.031847 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-online-tls\") pod \"6b5ba562-436b-491e-b635-bb1e15a58a0d\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " Sep 09 08:57:11.032101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.031884 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/60f30d66-5861-4cb8-83fd-f2844af8af40-feast-data\") pod \"60f30d66-5861-4cb8-83fd-f2844af8af40\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " Sep 09 08:57:11.032101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.031910 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls\") pod \"60f30d66-5861-4cb8-83fd-f2844af8af40\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " Sep 09 08:57:11.032101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.031940 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls\") pod \"60f30d66-5861-4cb8-83fd-f2844af8af40\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " Sep 09 08:57:11.032101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.031985 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpxgx\" (UniqueName: \"kubernetes.io/projected/60f30d66-5861-4cb8-83fd-f2844af8af40-kube-api-access-dpxgx\") pod \"60f30d66-5861-4cb8-83fd-f2844af8af40\" (UID: \"60f30d66-5861-4cb8-83fd-f2844af8af40\") " Sep 09 08:57:11.032101 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.032014 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-registry-tls\") pod \"6b5ba562-436b-491e-b635-bb1e15a58a0d\" (UID: \"6b5ba562-436b-491e-b635-bb1e15a58a0d\") " Sep 09 08:57:11.032859 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.032810 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f30d66-5861-4cb8-83fd-f2844af8af40-feast-data" (OuterVolumeSpecName: "feast-data") pod "60f30d66-5861-4cb8-83fd-f2844af8af40" (UID: "60f30d66-5861-4cb8-83fd-f2844af8af40"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:57:11.034582 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.034536 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b5ba562-436b-491e-b635-bb1e15a58a0d-kube-api-access-wqbwz" (OuterVolumeSpecName: "kube-api-access-wqbwz") pod "6b5ba562-436b-491e-b635-bb1e15a58a0d" (UID: "6b5ba562-436b-491e-b635-bb1e15a58a0d"). InnerVolumeSpecName "kube-api-access-wqbwz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:57:11.034686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.034648 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-online-tls" (OuterVolumeSpecName: "online-tls") pod "6b5ba562-436b-491e-b635-bb1e15a58a0d" (UID: "6b5ba562-436b-491e-b635-bb1e15a58a0d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:57:11.034686 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.034651 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f30d66-5861-4cb8-83fd-f2844af8af40-kube-api-access-dpxgx" (OuterVolumeSpecName: "kube-api-access-dpxgx") pod "60f30d66-5861-4cb8-83fd-f2844af8af40" (UID: "60f30d66-5861-4cb8-83fd-f2844af8af40"). InnerVolumeSpecName "kube-api-access-dpxgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 08:57:11.034783 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.034699 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "60f30d66-5861-4cb8-83fd-f2844af8af40" (UID: "60f30d66-5861-4cb8-83fd-f2844af8af40"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:57:11.034783 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.034719 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6b5ba562-436b-491e-b635-bb1e15a58a0d" (UID: "6b5ba562-436b-491e-b635-bb1e15a58a0d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:57:11.035403 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.035382 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls" (OuterVolumeSpecName: "online-tls") pod "60f30d66-5861-4cb8-83fd-f2844af8af40" (UID: "60f30d66-5861-4cb8-83fd-f2844af8af40"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 08:57:11.041506 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.041486 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b5ba562-436b-491e-b635-bb1e15a58a0d-feast-data" (OuterVolumeSpecName: "feast-data") pod "6b5ba562-436b-491e-b635-bb1e15a58a0d" (UID: "6b5ba562-436b-491e-b635-bb1e15a58a0d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 08:57:11.133601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.133570 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqbwz\" (UniqueName: \"kubernetes.io/projected/6b5ba562-436b-491e-b635-bb1e15a58a0d-kube-api-access-wqbwz\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:57:11.133601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.133595 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-online-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:57:11.133601 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.133605 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/60f30d66-5861-4cb8-83fd-f2844af8af40-feast-data\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:57:11.133871 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.133613 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-registry-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:57:11.133871 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.133620 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/60f30d66-5861-4cb8-83fd-f2844af8af40-online-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:57:11.133871 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.133628 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpxgx\" (UniqueName: \"kubernetes.io/projected/60f30d66-5861-4cb8-83fd-f2844af8af40-kube-api-access-dpxgx\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:57:11.133871 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.133636 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6b5ba562-436b-491e-b635-bb1e15a58a0d-registry-tls\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:57:11.133871 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.133644 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5ba562-436b-491e-b635-bb1e15a58a0d-feast-data\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 08:57:11.778047 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.778021 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" Sep 09 08:57:11.778495 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.778023 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq" event={"ID":"6b5ba562-436b-491e-b635-bb1e15a58a0d","Type":"ContainerDied","Data":"1af5b62ee9805a891828dfd186ec363e08dd2e5ecae00f9e64d32b0818eb2e00"} Sep 09 08:57:11.778495 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.778148 2577 scope.go:117] "RemoveContainer" containerID="53b6f2133ed33c525572254d88658168cbe8e3153caaa4ff8de40c0f470456f2" Sep 09 08:57:11.780568 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.780544 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" event={"ID":"60f30d66-5861-4cb8-83fd-f2844af8af40","Type":"ContainerDied","Data":"fca07c2f71d2f02fa7b46be910b8e3f658fa8275574df3ea815a3038db0ef365"} Sep 09 08:57:11.780679 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.780588 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv" Sep 09 08:57:11.789479 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.789459 2577 scope.go:117] "RemoveContainer" containerID="62e0ea325d6b33d1fbbc6f640f38a92eadc0529df591dbc7d52684a79bb6df4a" Sep 09 08:57:11.798399 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.798384 2577 scope.go:117] "RemoveContainer" containerID="638df6566311c9204df53c1b6d098f508ebbb8f92cb3152d52720a765ae163c5" Sep 09 08:57:11.805863 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.805841 2577 scope.go:117] "RemoveContainer" containerID="640ded2747c1ee5e6f2e1287b72e936404f8c19ab579e155b550fd129f8968cf" Sep 09 08:57:11.806317 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.806297 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq"] Sep 09 08:57:11.809716 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.809692 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-86f966697-2vgtq"] Sep 09 08:57:11.813529 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.813515 2577 scope.go:117] "RemoveContainer" containerID="ab7d84c86f03fc900ce9cd94a287644abe69eda65c79d77c695cebaeeb248ae5" Sep 09 08:57:11.819802 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.819783 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv"] Sep 09 08:57:11.820993 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.820965 2577 scope.go:117] "RemoveContainer" containerID="c3027137257c12149921365c20511cdd791f801cd52fc9190f1d348330f20ea1" Sep 09 08:57:11.823313 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.823295 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-57dbd78c8b-69sxv"] Sep 09 08:57:11.828690 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.828675 2577 scope.go:117] "RemoveContainer" containerID="be450ef953e47db9288152ed997a3c2a7982f7338275aa9a57fcd74b9f3e6290" Sep 09 08:57:11.836029 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:11.836012 2577 scope.go:117] "RemoveContainer" containerID="f01923cd02c624cc395e7904b1079c9845cf5a79a0ed0a30e254567e8fb823a6" Sep 09 08:57:12.634218 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:12.634187 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" path="/var/lib/kubelet/pods/60f30d66-5861-4cb8-83fd-f2844af8af40/volumes" Sep 09 08:57:12.634682 ip-10-0-133-59 kubenswrapper[2577]: I0909 08:57:12.634670 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" path="/var/lib/kubelet/pods/6b5ba562-436b-491e-b635-bb1e15a58a0d/volumes" Sep 09 09:08:42.180171 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180135 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5smsr/must-gather-wc4wh"] Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180479 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="feast-init" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180490 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="feast-init" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180500 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="registry" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180505 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="registry" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180512 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180518 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180525 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="feast-apply" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180530 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="feast-apply" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180537 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="feast-apply" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180542 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="feast-apply" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180547 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="feast-init" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180552 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="feast-init" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180561 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e7f29b2-cc1c-475e-be20-e92f9437777f" containerName="redis" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180566 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e7f29b2-cc1c-475e-be20-e92f9437777f" containerName="redis" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180574 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb9902ea-bc33-496a-bab8-676aa27027ce" containerName="postgres" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180579 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb9902ea-bc33-496a-bab8-676aa27027ce" containerName="postgres" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180587 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" Sep 09 09:08:42.180584 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180592 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" Sep 09 09:08:42.181282 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180600 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" Sep 09 09:08:42.181282 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180605 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" Sep 09 09:08:42.181282 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180652 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="online" Sep 09 09:08:42.181282 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180662 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e7f29b2-cc1c-475e-be20-e92f9437777f" containerName="redis" Sep 09 09:08:42.181282 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180667 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="registry" Sep 09 09:08:42.181282 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180673 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60f30d66-5861-4cb8-83fd-f2844af8af40" containerName="registry" Sep 09 09:08:42.181282 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180678 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb9902ea-bc33-496a-bab8-676aa27027ce" containerName="postgres" Sep 09 09:08:42.181282 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.180684 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b5ba562-436b-491e-b635-bb1e15a58a0d" containerName="online" Sep 09 09:08:42.183737 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.183722 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:08:42.186465 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.186440 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5smsr\"/\"default-dockercfg-xzjhv\"" Sep 09 09:08:42.186598 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.186478 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5smsr\"/\"openshift-service-ca.crt\"" Sep 09 09:08:42.186598 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.186498 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5smsr\"/\"kube-root-ca.crt\"" Sep 09 09:08:42.191270 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.191251 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5smsr/must-gather-wc4wh"] Sep 09 09:08:42.259724 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.259694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-must-gather-output\") pod \"must-gather-wc4wh\" (UID: \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\") " pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:08:42.259860 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.259734 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j6dv\" (UniqueName: \"kubernetes.io/projected/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-kube-api-access-5j6dv\") pod \"must-gather-wc4wh\" (UID: \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\") " pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:08:42.361052 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.361024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j6dv\" (UniqueName: \"kubernetes.io/projected/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-kube-api-access-5j6dv\") pod \"must-gather-wc4wh\" (UID: \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\") " pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:08:42.361190 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.361111 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-must-gather-output\") pod \"must-gather-wc4wh\" (UID: \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\") " pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:08:42.361367 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.361352 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-must-gather-output\") pod \"must-gather-wc4wh\" (UID: \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\") " pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:08:42.369881 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.369853 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j6dv\" (UniqueName: \"kubernetes.io/projected/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-kube-api-access-5j6dv\") pod \"must-gather-wc4wh\" (UID: \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\") " pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:08:42.501315 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.501293 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:08:42.772065 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.772043 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5smsr/must-gather-wc4wh"] Sep 09 09:08:42.773937 ip-10-0-133-59 kubenswrapper[2577]: W0909 09:08:42.773911 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0dcd1ec_778d_488c_adf6_e3dc397b2e45.slice/crio-23d95923d3cb48256a6df7c448df8b44b561a92eb196845909006d2e2ab08735 WatchSource:0}: Error finding container 23d95923d3cb48256a6df7c448df8b44b561a92eb196845909006d2e2ab08735: Status 404 returned error can't find the container with id 23d95923d3cb48256a6df7c448df8b44b561a92eb196845909006d2e2ab08735 Sep 09 09:08:42.775759 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:42.775742 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 09:08:43.581092 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:43.581058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5smsr/must-gather-wc4wh" event={"ID":"e0dcd1ec-778d-488c-adf6-e3dc397b2e45","Type":"ContainerStarted","Data":"23d95923d3cb48256a6df7c448df8b44b561a92eb196845909006d2e2ab08735"} Sep 09 09:08:50.620903 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:50.620861 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5smsr/must-gather-wc4wh" event={"ID":"e0dcd1ec-778d-488c-adf6-e3dc397b2e45","Type":"ContainerStarted","Data":"04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c"} Sep 09 09:08:50.621328 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:50.620911 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5smsr/must-gather-wc4wh" event={"ID":"e0dcd1ec-778d-488c-adf6-e3dc397b2e45","Type":"ContainerStarted","Data":"2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8"} Sep 09 09:08:50.637068 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:50.637013 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5smsr/must-gather-wc4wh" podStartSLOduration=1.741070007 podStartE2EDuration="8.636992889s" totalImagesPullingTime="6.895922882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 09:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 09:08:50.636536542 +0000 UTC m=+1774.469900103" watchObservedRunningTime="2026-09-09 09:08:50.636992889 +0000 UTC m=+1774.470356447" Sep 09 09:08:59.663544 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:59.663451 2577 generic.go:356] "Generic (PLEG): container finished" podID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerID="2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8" exitCode=0 Sep 09 09:08:59.663544 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:59.663525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5smsr/must-gather-wc4wh" event={"ID":"e0dcd1ec-778d-488c-adf6-e3dc397b2e45","Type":"ContainerDied","Data":"2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8"} Sep 09 09:08:59.664028 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:08:59.663843 2577 scope.go:117] "RemoveContainer" containerID="2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8" Sep 09 09:09:00.284464 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:00.284433 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-5smsr_must-gather-wc4wh_e0dcd1ec-778d-488c-adf6-e3dc397b2e45/gather/0.log" Sep 09 09:09:03.595452 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:03.595420 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c25ds_6e9b855f-9ef8-4c2f-903d-492a4119d176/global-pull-secret-syncer/0.log" Sep 09 09:09:03.762994 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:03.762943 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9qxkj_e4e33c67-cd89-4b31-946f-0792b3e9a131/konnectivity-agent/0.log" Sep 09 09:09:03.841221 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:03.841190 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-59.ec2.internal_581b21db7cc54e6fc2cafbc222baf6c8/haproxy/0.log" Sep 09 09:09:05.605698 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.605659 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-5smsr/must-gather-wc4wh"] Sep 09 09:09:05.606190 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.605917 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-5smsr/must-gather-wc4wh" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerName="copy" containerID="cri-o://04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c" gracePeriod=2 Sep 09 09:09:05.611919 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.611889 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-5smsr/must-gather-wc4wh"] Sep 09 09:09:05.613157 ip-10-0-133-59 kubenswrapper[2577]: E0909 09:09:05.613121 2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-5smsr\"/\"must-gather-wc4wh\": pods \"must-gather-wc4wh\" not found" pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:09:05.841354 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.841330 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-5smsr_must-gather-wc4wh_e0dcd1ec-778d-488c-adf6-e3dc397b2e45/copy/0.log" Sep 09 09:09:05.842956 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.842938 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:09:05.845410 ip-10-0-133-59 kubenswrapper[2577]: E0909 09:09:05.845383 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-wc4wh\" is forbidden: User \"system:node:ip-10-0-133-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-5smsr\": no relationship found between node 'ip-10-0-133-59.ec2.internal' and this object" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:09:05.868190 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.868136 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-must-gather-output\") pod \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\" (UID: \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\") " Sep 09 09:09:05.868190 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.868173 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j6dv\" (UniqueName: \"kubernetes.io/projected/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-kube-api-access-5j6dv\") pod \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\" (UID: \"e0dcd1ec-778d-488c-adf6-e3dc397b2e45\") " Sep 09 09:09:05.868465 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.868445 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "e0dcd1ec-778d-488c-adf6-e3dc397b2e45" (UID: "e0dcd1ec-778d-488c-adf6-e3dc397b2e45"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 09:09:05.870251 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.870225 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-kube-api-access-5j6dv" (OuterVolumeSpecName: "kube-api-access-5j6dv") pod "e0dcd1ec-778d-488c-adf6-e3dc397b2e45" (UID: "e0dcd1ec-778d-488c-adf6-e3dc397b2e45"). InnerVolumeSpecName "kube-api-access-5j6dv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 09:09:05.969322 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.969273 2577 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-must-gather-output\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 09:09:05.969322 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:05.969318 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j6dv\" (UniqueName: \"kubernetes.io/projected/e0dcd1ec-778d-488c-adf6-e3dc397b2e45-kube-api-access-5j6dv\") on node \"ip-10-0-133-59.ec2.internal\" DevicePath \"\"" Sep 09 09:09:06.632573 ip-10-0-133-59 kubenswrapper[2577]: E0909 09:09:06.632531 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-wc4wh\" is forbidden: User \"system:node:ip-10-0-133-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-5smsr\": no relationship found between node 'ip-10-0-133-59.ec2.internal' and this object" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:09:06.633865 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.633843 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" path="/var/lib/kubelet/pods/e0dcd1ec-778d-488c-adf6-e3dc397b2e45/volumes" Sep 09 09:09:06.695448 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.695422 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-5smsr_must-gather-wc4wh_e0dcd1ec-778d-488c-adf6-e3dc397b2e45/copy/0.log" Sep 09 09:09:06.695786 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.695764 2577 generic.go:356] "Generic (PLEG): container finished" podID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerID="04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c" exitCode=143 Sep 09 09:09:06.695838 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.695817 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5smsr/must-gather-wc4wh" Sep 09 09:09:06.695838 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.695836 2577 scope.go:117] "RemoveContainer" containerID="04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c" Sep 09 09:09:06.703316 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.703298 2577 scope.go:117] "RemoveContainer" containerID="2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8" Sep 09 09:09:06.715093 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.715076 2577 scope.go:117] "RemoveContainer" containerID="04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c" Sep 09 09:09:06.715389 ip-10-0-133-59 kubenswrapper[2577]: E0909 09:09:06.715371 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c\": container with ID starting with 04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c not found: ID does not exist" containerID="04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c" Sep 09 09:09:06.715442 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.715401 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c"} err="failed to get container status \"04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c\": rpc error: code = NotFound desc = could not find container \"04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c\": container with ID starting with 04c5590179adf5f362de6f1e14d76a8915429ae8db62cc2898ca8420471ed77c not found: ID does not exist" Sep 09 09:09:06.715442 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.715419 2577 scope.go:117] "RemoveContainer" containerID="2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8" Sep 09 09:09:06.715676 ip-10-0-133-59 kubenswrapper[2577]: E0909 09:09:06.715658 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8\": container with ID starting with 2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8 not found: ID does not exist" containerID="2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8" Sep 09 09:09:06.715732 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:06.715686 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8"} err="failed to get container status \"2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8\": rpc error: code = NotFound desc = could not find container \"2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8\": container with ID starting with 2e87c92f641e571e5306aa652c80734b69635d8f3b9d60be06b49b793271aed8 not found: ID does not exist" Sep 09 09:09:07.281035 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.281009 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0792c64b-7c8b-4dd8-b329-2fb6919eb43d/alertmanager/0.log" Sep 09 09:09:07.308604 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.308577 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0792c64b-7c8b-4dd8-b329-2fb6919eb43d/config-reloader/0.log" Sep 09 09:09:07.334942 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.334921 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0792c64b-7c8b-4dd8-b329-2fb6919eb43d/kube-rbac-proxy-web/0.log" Sep 09 09:09:07.359559 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.359533 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0792c64b-7c8b-4dd8-b329-2fb6919eb43d/kube-rbac-proxy/0.log" Sep 09 09:09:07.385003 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.384981 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0792c64b-7c8b-4dd8-b329-2fb6919eb43d/kube-rbac-proxy-metric/0.log" Sep 09 09:09:07.420839 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.420813 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0792c64b-7c8b-4dd8-b329-2fb6919eb43d/prom-label-proxy/0.log" Sep 09 09:09:07.445985 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.445949 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0792c64b-7c8b-4dd8-b329-2fb6919eb43d/init-config-reloader/0.log" Sep 09 09:09:07.528676 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.528647 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-kfdk4_740bc940-3f52-4ad9-8a98-ad998acdd9a0/kube-state-metrics/0.log" Sep 09 09:09:07.551532 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.551469 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-kfdk4_740bc940-3f52-4ad9-8a98-ad998acdd9a0/kube-rbac-proxy-main/0.log" Sep 09 09:09:07.574945 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.574920 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-kfdk4_740bc940-3f52-4ad9-8a98-ad998acdd9a0/kube-rbac-proxy-self/0.log" Sep 09 09:09:07.608412 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.608388 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7dcb6bdff4-5gj4z_0f65a43c-1489-42ce-a0e7-38aa7b1d7fcc/metrics-server/0.log" Sep 09 09:09:07.643402 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.643376 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-nfs8l_170864f0-3e0b-4d2e-a971-2edad85e5df2/monitoring-plugin/0.log" Sep 09 09:09:07.674858 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.674832 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kp2sc_c61504b7-2791-49ff-9631-c5ef7b12c6a8/node-exporter/0.log" Sep 09 09:09:07.696169 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.696148 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kp2sc_c61504b7-2791-49ff-9631-c5ef7b12c6a8/kube-rbac-proxy/0.log" Sep 09 09:09:07.720861 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:07.720835 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kp2sc_c61504b7-2791-49ff-9631-c5ef7b12c6a8/init-textfile/0.log" Sep 09 09:09:08.009701 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.009675 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4cg7q_1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6/kube-rbac-proxy-main/0.log" Sep 09 09:09:08.031982 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.031940 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4cg7q_1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6/kube-rbac-proxy-self/0.log" Sep 09 09:09:08.057050 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.057025 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4cg7q_1d3b1ffe-3f4e-4ff6-b892-624a0f15e3a6/openshift-state-metrics/0.log" Sep 09 09:09:08.108669 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.108639 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_177f7594-7af0-4d01-b453-4a15988ff6ca/prometheus/0.log" Sep 09 09:09:08.144460 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.144433 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_177f7594-7af0-4d01-b453-4a15988ff6ca/config-reloader/0.log" Sep 09 09:09:08.168764 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.168739 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_177f7594-7af0-4d01-b453-4a15988ff6ca/thanos-sidecar/0.log" Sep 09 09:09:08.190301 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.190275 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_177f7594-7af0-4d01-b453-4a15988ff6ca/kube-rbac-proxy-web/0.log" Sep 09 09:09:08.213240 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.213214 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_177f7594-7af0-4d01-b453-4a15988ff6ca/kube-rbac-proxy/0.log" Sep 09 09:09:08.236604 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.236585 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_177f7594-7af0-4d01-b453-4a15988ff6ca/kube-rbac-proxy-thanos/0.log" Sep 09 09:09:08.259784 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.259718 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_177f7594-7af0-4d01-b453-4a15988ff6ca/init-config-reloader/0.log" Sep 09 09:09:08.287790 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.287768 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-2rt5x_fee83f8d-c796-4c40-a05d-bb5eb71c4531/prometheus-operator/0.log" Sep 09 09:09:08.312060 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.312020 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-2rt5x_fee83f8d-c796-4c40-a05d-bb5eb71c4531/kube-rbac-proxy/0.log" Sep 09 09:09:08.341842 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.341819 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-c7qkw_a957469c-2738-4ce2-b169-39432f7394f0/prometheus-operator-admission-webhook/0.log" Sep 09 09:09:08.378055 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.378036 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9b4698bf4-sldlc_45773a22-188c-4017-be16-31e2f3d4714b/telemeter-client/0.log" Sep 09 09:09:08.400239 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.400218 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9b4698bf4-sldlc_45773a22-188c-4017-be16-31e2f3d4714b/reload/0.log" Sep 09 09:09:08.423515 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.423485 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9b4698bf4-sldlc_45773a22-188c-4017-be16-31e2f3d4714b/kube-rbac-proxy/0.log" Sep 09 09:09:08.455450 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.455423 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76bcd6d8d4-kbr4s_dc7027bc-de24-49d0-bc2a-ff7bba8f75b5/thanos-query/0.log" Sep 09 09:09:08.476689 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.476664 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76bcd6d8d4-kbr4s_dc7027bc-de24-49d0-bc2a-ff7bba8f75b5/kube-rbac-proxy-web/0.log" Sep 09 09:09:08.496597 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.496569 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76bcd6d8d4-kbr4s_dc7027bc-de24-49d0-bc2a-ff7bba8f75b5/kube-rbac-proxy/0.log" Sep 09 09:09:08.516194 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.516131 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76bcd6d8d4-kbr4s_dc7027bc-de24-49d0-bc2a-ff7bba8f75b5/prom-label-proxy/0.log" Sep 09 09:09:08.536162 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.536139 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76bcd6d8d4-kbr4s_dc7027bc-de24-49d0-bc2a-ff7bba8f75b5/kube-rbac-proxy-rules/0.log" Sep 09 09:09:08.558384 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:08.558360 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76bcd6d8d4-kbr4s_dc7027bc-de24-49d0-bc2a-ff7bba8f75b5/kube-rbac-proxy-metrics/0.log" Sep 09 09:09:10.703148 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:10.703119 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-4rmzn_743d622f-3fd6-4d52-be02-fe88084a7cc2/download-server/0.log" Sep 09 09:09:11.286690 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.286658 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb"] Sep 09 09:09:11.287010 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.286998 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerName="copy" Sep 09 09:09:11.287067 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.287011 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerName="copy" Sep 09 09:09:11.287067 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.287026 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerName="gather" Sep 09 09:09:11.287067 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.287032 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerName="gather" Sep 09 09:09:11.287158 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.287086 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerName="gather" Sep 09 09:09:11.287158 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.287097 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0dcd1ec-778d-488c-adf6-e3dc397b2e45" containerName="copy" Sep 09 09:09:11.291194 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.291175 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.295856 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.295834 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pnrnn\"/\"kube-root-ca.crt\"" Sep 09 09:09:11.296817 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.296794 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pnrnn\"/\"default-dockercfg-hm2qb\"" Sep 09 09:09:11.296942 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.296794 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pnrnn\"/\"openshift-service-ca.crt\"" Sep 09 09:09:11.302503 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.302483 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb"] Sep 09 09:09:11.416820 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.416782 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-proc\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.417078 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.416841 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-sys\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.417078 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.416930 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-podres\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.417078 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.417000 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfn7j\" (UniqueName: \"kubernetes.io/projected/5be3a19c-17b7-4694-b1cb-b0112d3fce66-kube-api-access-vfn7j\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.417078 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.417033 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-lib-modules\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517542 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-proc\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517723 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-sys\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517723 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-podres\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517723 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517621 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfn7j\" (UniqueName: \"kubernetes.io/projected/5be3a19c-17b7-4694-b1cb-b0112d3fce66-kube-api-access-vfn7j\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517723 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-lib-modules\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517723 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517639 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-proc\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517955 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517728 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-sys\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517955 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517763 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-podres\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.517955 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.517776 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5be3a19c-17b7-4694-b1cb-b0112d3fce66-lib-modules\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.526922 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.526899 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfn7j\" (UniqueName: \"kubernetes.io/projected/5be3a19c-17b7-4694-b1cb-b0112d3fce66-kube-api-access-vfn7j\") pod \"perf-node-gather-daemonset-v8kqb\" (UID: \"5be3a19c-17b7-4694-b1cb-b0112d3fce66\") " pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.600716 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.600626 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:11.871533 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:11.871411 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb"] Sep 09 09:09:11.873793 ip-10-0-133-59 kubenswrapper[2577]: W0909 09:09:11.873759 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5be3a19c_17b7_4694_b1cb_b0112d3fce66.slice/crio-4759386239d59d44d4c1f8ad0dce877f83d8076125499c27795110c7fe5c06e2 WatchSource:0}: Error finding container 4759386239d59d44d4c1f8ad0dce877f83d8076125499c27795110c7fe5c06e2: Status 404 returned error can't find the container with id 4759386239d59d44d4c1f8ad0dce877f83d8076125499c27795110c7fe5c06e2 Sep 09 09:09:12.125511 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:12.125425 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c9gmd_6bb6dfa7-85ee-487b-bec3-e3e99c71a526/dns/0.log" Sep 09 09:09:12.144786 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:12.144764 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c9gmd_6bb6dfa7-85ee-487b-bec3-e3e99c71a526/kube-rbac-proxy/0.log" Sep 09 09:09:12.238590 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:12.238562 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lcbk4_7420c89d-6441-4350-9c6a-6a5b3fd8c2ca/dns-node-resolver/0.log" Sep 09 09:09:12.672094 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:12.672065 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jmn4t_333ce92a-033d-45bc-bbfc-78ea69ff251c/node-ca/0.log" Sep 09 09:09:12.721778 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:12.721740 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" event={"ID":"5be3a19c-17b7-4694-b1cb-b0112d3fce66","Type":"ContainerStarted","Data":"cdb8ed104d2a01114979b881c8075f882538e8b01d5cca87e5c05fa50b1a3501"} Sep 09 09:09:12.721943 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:12.721791 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" event={"ID":"5be3a19c-17b7-4694-b1cb-b0112d3fce66","Type":"ContainerStarted","Data":"4759386239d59d44d4c1f8ad0dce877f83d8076125499c27795110c7fe5c06e2"} Sep 09 09:09:12.721943 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:12.721816 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:12.737674 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:12.737621 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" podStartSLOduration=1.7376081239999999 podStartE2EDuration="1.737608124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 09:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 09:09:12.735693067 +0000 UTC m=+1796.569056641" watchObservedRunningTime="2026-09-09 09:09:12.737608124 +0000 UTC m=+1796.570971713" Sep 09 09:09:13.698611 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:13.698580 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s7mrr_294058ff-720d-4cb7-a1da-cece1197a475/serve-healthcheck-canary/0.log" Sep 09 09:09:14.062157 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:14.062129 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7btxn_12fd7ed4-407d-4fa2-854b-a483c8d81072/kube-rbac-proxy/0.log" Sep 09 09:09:14.083013 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:14.082987 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7btxn_12fd7ed4-407d-4fa2-854b-a483c8d81072/exporter/0.log" Sep 09 09:09:14.103295 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:14.103270 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7btxn_12fd7ed4-407d-4fa2-854b-a483c8d81072/extractor/0.log" Sep 09 09:09:18.735990 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:18.735947 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pnrnn/perf-node-gather-daemonset-v8kqb" Sep 09 09:09:19.355312 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.355285 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q76kl_4b081bd8-2c98-4c0f-894b-3f8af9f71dbd/kube-multus-additional-cni-plugins/0.log" Sep 09 09:09:19.376099 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.376073 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q76kl_4b081bd8-2c98-4c0f-894b-3f8af9f71dbd/egress-router-binary-copy/0.log" Sep 09 09:09:19.396673 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.396652 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q76kl_4b081bd8-2c98-4c0f-894b-3f8af9f71dbd/cni-plugins/0.log" Sep 09 09:09:19.419151 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.419129 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q76kl_4b081bd8-2c98-4c0f-894b-3f8af9f71dbd/bond-cni-plugin/0.log" Sep 09 09:09:19.457455 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.457427 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q76kl_4b081bd8-2c98-4c0f-894b-3f8af9f71dbd/routeoverride-cni/0.log" Sep 09 09:09:19.479276 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.479256 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q76kl_4b081bd8-2c98-4c0f-894b-3f8af9f71dbd/whereabouts-cni-bincopy/0.log" Sep 09 09:09:19.501982 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.501960 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q76kl_4b081bd8-2c98-4c0f-894b-3f8af9f71dbd/whereabouts-cni/0.log" Sep 09 09:09:19.708204 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.708134 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cqjwk_65a71213-3745-4fe0-8fd1-372771761d7a/kube-multus/0.log" Sep 09 09:09:19.880083 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.880052 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wqxmh_a4992530-e8b6-4ebd-bc09-04038870dfbf/network-metrics-daemon/0.log" Sep 09 09:09:19.899603 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:19.899575 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wqxmh_a4992530-e8b6-4ebd-bc09-04038870dfbf/kube-rbac-proxy/0.log" Sep 09 09:09:21.114356 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:21.114328 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jw7m5_affb99ae-08fe-4af4-8b48-6492cc1fd4e1/ovn-controller/0.log" Sep 09 09:09:21.132463 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:21.132440 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jw7m5_affb99ae-08fe-4af4-8b48-6492cc1fd4e1/ovn-acl-logging/0.log" Sep 09 09:09:21.152668 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:21.152643 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jw7m5_affb99ae-08fe-4af4-8b48-6492cc1fd4e1/kube-rbac-proxy-node/0.log" Sep 09 09:09:21.173597 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:21.173577 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jw7m5_affb99ae-08fe-4af4-8b48-6492cc1fd4e1/kube-rbac-proxy-ovn-metrics/0.log" Sep 09 09:09:21.193563 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:21.193531 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jw7m5_affb99ae-08fe-4af4-8b48-6492cc1fd4e1/northd/0.log" Sep 09 09:09:21.213933 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:21.213915 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jw7m5_affb99ae-08fe-4af4-8b48-6492cc1fd4e1/nbdb/0.log" Sep 09 09:09:21.234339 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:21.234320 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jw7m5_affb99ae-08fe-4af4-8b48-6492cc1fd4e1/sbdb/0.log" Sep 09 09:09:21.269256 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:21.269233 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jw7m5_affb99ae-08fe-4af4-8b48-6492cc1fd4e1/ovnkube-controller/0.log" Sep 09 09:09:22.311374 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:22.311335 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-9w6g4_dbeeae9e-b538-462d-a818-c8914cfdc325/network-check-target-container/0.log" Sep 09 09:09:23.224347 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:23.224317 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-tw4wg_6a228400-e2ea-43dc-85c0-9bf945bb0872/iptables-alerter/0.log" Sep 09 09:09:23.817646 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:23.817616 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-tvfvr_c4cf79c6-a064-4f3c-a9a8-204c611fa14e/tuned/0.log" Sep 09 09:09:27.222935 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:27.222908 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-mwnx6_7f562836-0b51-4578-a0d0-df5bba3702c8/csi-driver/0.log" Sep 09 09:09:27.242703 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:27.242677 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-mwnx6_7f562836-0b51-4578-a0d0-df5bba3702c8/csi-node-driver-registrar/0.log" Sep 09 09:09:27.261723 ip-10-0-133-59 kubenswrapper[2577]: I0909 09:09:27.261693 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-mwnx6_7f562836-0b51-4578-a0d0-df5bba3702c8/csi-liveness-probe/0.log"