Jun 03 16:14:26.045018 ip-10-0-139-61 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 03 16:14:26.045034 ip-10-0-139-61 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 03 16:14:26.045043 ip-10-0-139-61 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 03 16:14:26.045388 ip-10-0-139-61 systemd[1]: Failed to start Kubernetes Kubelet. Jun 03 16:14:36.095189 ip-10-0-139-61 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 03 16:14:36.095206 ip-10-0-139-61 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot bfd2b562a55f418bb57dfb5763ff2f13 -- Jun 03 16:17:15.586292 ip-10-0-139-61 systemd[1]: Starting Kubernetes Kubelet... Jun 03 16:17:16.037355 ip-10-0-139-61 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:17:16.037355 ip-10-0-139-61 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 16:17:16.037355 ip-10-0-139-61 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:17:16.037355 ip-10-0-139-61 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 16:17:16.037355 ip-10-0-139-61 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:17:16.039092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.038998 2563 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 16:17:16.042172 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042156 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:16.042172 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042170 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042174 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042179 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042182 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042185 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042188 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042191 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042194 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042196 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042199 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042202 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042205 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042208 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042211 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042214 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042216 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042225 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042229 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042232 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042235 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:16.042236 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042238 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042240 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042243 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042246 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042249 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042252 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042254 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042256 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042260 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042262 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042264 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042267 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042269 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042272 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042274 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042276 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042279 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042281 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042284 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042286 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:16.042717 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042289 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042291 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042294 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042296 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042299 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042301 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042304 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042306 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042309 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042312 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042315 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042317 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042320 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042322 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042326 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042335 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042338 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042340 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042343 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:16.043253 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042346 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042348 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042351 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042355 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042358 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042360 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042363 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042365 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042367 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042370 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042374 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042376 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042379 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042381 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042384 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042387 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042389 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042391 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042394 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042397 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:16.043708 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042400 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042402 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042405 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042409 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042412 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042415 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042788 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042793 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042796 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042799 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042803 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042806 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042809 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042812 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042815 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042824 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042827 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042830 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042833 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:16.044214 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042835 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042838 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042841 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042844 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042846 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042850 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042854 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042857 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042860 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042862 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042865 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042867 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042870 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042873 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042875 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042878 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042880 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042883 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042886 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:16.044660 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042888 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042891 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042894 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042897 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042899 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042902 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042905 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042908 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042910 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042913 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042915 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042917 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042920 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042922 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042925 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042927 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042930 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042933 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042935 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042938 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:16.045146 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042940 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042956 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042959 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042961 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042964 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042967 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042970 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042972 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042975 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042977 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042980 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042983 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042985 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042988 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042990 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042993 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042995 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.042998 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043000 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043003 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:16.045637 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043005 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043008 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043010 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043012 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043015 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043018 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043020 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043023 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043025 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043027 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043033 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043035 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043039 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043042 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043135 2563 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043146 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043157 2563 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043164 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043170 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043173 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043177 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 16:17:16.046147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043182 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043185 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043188 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043191 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043195 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043198 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043201 2563 flags.go:64] FLAG: --cgroup-root="" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043203 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043206 2563 flags.go:64] FLAG: --client-ca-file="" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043209 2563 flags.go:64] FLAG: --cloud-config="" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043212 2563 flags.go:64] FLAG: --cloud-provider="external" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043215 2563 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043219 2563 flags.go:64] FLAG: --cluster-domain="" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043222 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043225 2563 flags.go:64] FLAG: --config-dir="" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043228 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043231 2563 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043235 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043237 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043241 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043244 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043249 2563 flags.go:64] FLAG: --contention-profiling="false" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043253 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043256 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043259 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 16:17:16.046743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043263 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043267 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043270 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043273 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043276 2563 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043279 2563 flags.go:64] FLAG: --enable-server="true" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043282 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043287 2563 flags.go:64] FLAG: --event-burst="100" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043290 2563 flags.go:64] FLAG: --event-qps="50" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043293 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043296 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043299 2563 flags.go:64] FLAG: --eviction-hard="" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043302 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043305 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043308 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043311 2563 flags.go:64] FLAG: --eviction-soft="" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043314 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043316 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043319 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043322 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043325 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043328 2563 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043330 2563 flags.go:64] FLAG: --feature-gates="" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043334 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043337 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 16:17:16.047364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043341 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043345 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043349 2563 flags.go:64] FLAG: --healthz-port="10248" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043353 2563 flags.go:64] FLAG: --help="false" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043355 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043359 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043362 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043365 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043368 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043371 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043373 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043376 2563 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043379 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043382 2563 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043384 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043388 2563 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043390 2563 flags.go:64] FLAG: --kube-reserved="" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043393 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043396 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043399 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043402 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043405 2563 flags.go:64] FLAG: --lock-file="" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043407 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043410 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 16:17:16.048003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043413 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043418 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043421 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043424 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043426 2563 flags.go:64] FLAG: --logging-format="text" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043429 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043432 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043435 2563 flags.go:64] FLAG: --manifest-url="" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043438 2563 flags.go:64] FLAG: --manifest-url-header="" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043443 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043446 2563 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043452 2563 flags.go:64] FLAG: --max-pods="110" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043454 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043458 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043461 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043464 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043467 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043469 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043472 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043480 2563 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043483 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043486 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043489 2563 flags.go:64] FLAG: --pod-cidr="" Jun 03 16:17:16.048594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043492 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043498 2563 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043501 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043504 2563 flags.go:64] FLAG: --pods-per-core="0" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043506 2563 flags.go:64] FLAG: --port="10250" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043509 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043512 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-078441adc63b97d1d" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043515 2563 flags.go:64] FLAG: --qos-reserved="" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043518 2563 flags.go:64] FLAG: --read-only-port="10255" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043520 2563 flags.go:64] FLAG: --register-node="true" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043523 2563 flags.go:64] FLAG: --register-schedulable="true" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043526 2563 flags.go:64] FLAG: --register-with-taints="" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043536 2563 flags.go:64] FLAG: --registry-burst="10" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043539 2563 flags.go:64] FLAG: --registry-qps="5" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043542 2563 flags.go:64] FLAG: --reserved-cpus="" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043544 2563 flags.go:64] FLAG: --reserved-memory="" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043548 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043551 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043555 2563 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043558 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043560 2563 flags.go:64] FLAG: --runonce="false" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043564 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043567 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043570 2563 flags.go:64] FLAG: --seccomp-default="false" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043574 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043577 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 16:17:16.049176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043580 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043583 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043586 2563 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043589 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043591 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043594 2563 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043597 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043600 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043603 2563 flags.go:64] FLAG: --system-cgroups="" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043605 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043610 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043613 2563 flags.go:64] FLAG: --tls-cert-file="" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043616 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043620 2563 flags.go:64] FLAG: --tls-min-version="" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043623 2563 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043625 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043628 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043631 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043633 2563 flags.go:64] FLAG: --v="2" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043638 2563 flags.go:64] FLAG: --version="false" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043641 2563 flags.go:64] FLAG: --vmodule="" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043646 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.043649 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043747 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043752 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:16.049792 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043755 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043758 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043763 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043766 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043768 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043772 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043775 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043778 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043781 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043784 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043786 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043789 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043791 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043794 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043797 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043799 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043802 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043804 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043807 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:16.050441 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043809 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043811 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043814 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043816 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043819 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043821 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043824 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043827 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043829 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043832 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043834 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043836 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043840 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043843 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043845 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043849 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043852 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043854 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043857 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043859 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:16.051268 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043862 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043865 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043867 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043870 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043872 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043875 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043877 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043881 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043884 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043887 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043890 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043893 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043896 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043899 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043901 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043904 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043906 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043909 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043911 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043913 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:16.052228 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043916 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043918 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043921 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043923 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043927 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043929 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043932 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043935 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043938 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043941 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043958 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043961 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043963 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043966 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043969 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043972 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043976 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043979 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043982 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:16.053150 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043984 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043987 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043990 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043992 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043995 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.043998 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.044788 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.052862 2563 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.052885 2563 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.052987 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.052997 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053001 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053007 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053012 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053016 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053020 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:16.053996 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053024 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053029 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053033 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053037 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053041 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053046 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053050 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053054 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053058 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053063 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053067 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053080 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053085 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053089 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053094 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053098 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053102 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053106 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053110 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053114 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:16.054508 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053118 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053123 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053127 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053131 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053146 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053151 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053155 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053159 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053163 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053166 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053170 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053175 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053179 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053183 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053187 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053191 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053195 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053199 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053203 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053210 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:16.055069 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053221 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053227 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053232 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053236 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053241 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053245 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053249 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053253 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053257 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053261 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053265 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053269 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053274 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053277 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053282 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053286 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053291 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053304 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053308 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053313 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:16.055583 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053317 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053321 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053327 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053334 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053338 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053342 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053346 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053350 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053354 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053359 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053363 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053367 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053372 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053376 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053380 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053384 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053388 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053391 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:16.056309 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053396 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.053404 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053616 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053625 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053630 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053635 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053639 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053643 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053647 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053652 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053657 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053661 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053673 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053678 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053682 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:16.057129 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053686 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053690 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053694 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053698 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053703 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053707 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053711 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053715 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053719 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053722 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053727 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053731 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053735 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053739 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053743 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053747 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053753 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053760 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053764 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:16.057653 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053767 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053772 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053776 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053780 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053784 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053788 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053791 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053796 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053801 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053805 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053809 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053822 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053826 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053830 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053834 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053838 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053842 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053846 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053851 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:16.058163 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053855 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053860 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053864 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053868 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053872 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053876 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053880 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053884 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053888 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053892 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053895 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053899 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053903 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053907 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053911 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053915 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053919 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053923 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053927 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053931 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:16.058754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053935 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053939 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053965 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053970 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053974 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053987 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053991 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053995 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.053999 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.054002 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.054006 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.054010 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.054016 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.054021 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:16.054026 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:16.059272 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.054035 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:17:16.059699 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.054969 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 16:17:16.059699 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.058228 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 16:17:16.059699 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.059411 2563 server.go:1019] "Starting client certificate rotation" Jun 03 16:17:16.059699 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.059517 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 16:17:16.060875 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.060858 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 16:17:16.088673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.088633 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 16:17:16.090692 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.090655 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 16:17:16.106475 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.106451 2563 log.go:25] "Validated CRI v1 runtime API" Jun 03 16:17:16.114172 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.114154 2563 log.go:25] "Validated CRI v1 image API" Jun 03 16:17:16.116902 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.116883 2563 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 16:17:16.122910 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.122887 2563 fs.go:135] Filesystem UUIDs: map[1cae9a7d-e46d-49d4-be1c-bcc6394e7107:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9778fba5-785b-4130-8739-f484b74368c6:/dev/nvme0n1p3] Jun 03 16:17:16.122995 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.122910 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 16:17:16.124928 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.124910 2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 16:17:16.127595 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.127489 2563 manager.go:217] Machine: {Timestamp:2026-06-03 16:17:16.126996801 +0000 UTC m=+0.422189833 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095874 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b76ab541ca90802ca60be9b0aa5b3 SystemUUID:ec2b76ab-541c-a908-02ca-60be9b0aa5b3 BootID:bfd2b562-a55f-418b-b57d-fb5763ff2f13 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9e:71:d1:02:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9e:71:d1:02:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:3f:bb:50:e2:51 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 16:17:16.128289 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.128278 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 16:17:16.128376 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.128364 2563 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 16:17:16.130570 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.130541 2563 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 16:17:16.130743 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.130573 2563 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-61.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 16:17:16.130828 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.130759 2563 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 16:17:16.130828 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.130771 2563 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 16:17:16.130828 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.130781 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 16:17:16.132895 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.132881 2563 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 16:17:16.134198 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.134184 2563 state_mem.go:36] "Initialized new in-memory state store" Jun 03 16:17:16.134335 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.134324 2563 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 16:17:16.136780 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.136768 2563 kubelet.go:491] "Attempting to sync node with API server" Jun 03 16:17:16.136836 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.136808 2563 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 16:17:16.136836 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.136827 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 16:17:16.136925 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.136841 2563 kubelet.go:397] "Adding apiserver pod source" Jun 03 16:17:16.136925 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.136854 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 16:17:16.137894 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.137880 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 16:17:16.137987 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.137905 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 16:17:16.144935 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.144913 2563 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 16:17:16.150147 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.150033 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-crdrh" Jun 03 16:17:16.150258 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.150174 2563 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 16:17:16.150357 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.150322 2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 16:17:16.150836 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.150807 2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 16:17:16.152746 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152730 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152750 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152756 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152762 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152770 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152776 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152781 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152786 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152793 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152799 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152808 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 16:17:16.152810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.152814 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 16:17:16.154514 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.154503 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 16:17:16.154514 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.154515 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 16:17:16.156940 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.156921 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-crdrh" Jun 03 16:17:16.158284 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.158271 2563 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 16:17:16.158333 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.158309 2563 server.go:1295] "Started kubelet" Jun 03 16:17:16.158454 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.158394 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 16:17:16.158491 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.158418 2563 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 16:17:16.158551 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.158537 2563 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 16:17:16.159323 ip-10-0-139-61 systemd[1]: Started Kubernetes Kubelet. Jun 03 16:17:16.161231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.161214 2563 server.go:317] "Adding debug handlers to kubelet server" Jun 03 16:17:16.162900 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.162882 2563 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 16:17:16.168695 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.168664 2563 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 16:17:16.168799 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.168766 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 16:17:16.169314 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.169299 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 16:17:16.169394 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.169384 2563 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 16:17:16.169394 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.169394 2563 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 16:17:16.169526 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.169513 2563 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 16:17:16.169580 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.169561 2563 reconstruct.go:97] "Volume reconstruction finished" Jun 03 16:17:16.169580 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.169567 2563 reconciler.go:26] "Reconciler: start to sync state" Jun 03 16:17:16.170075 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.170053 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.170297 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.170284 2563 factory.go:153] Registering CRI-O factory Jun 03 16:17:16.170363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.170302 2563 factory.go:223] Registration of the crio container factory successfully Jun 03 16:17:16.170421 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.170362 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 16:17:16.170421 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.170372 2563 factory.go:55] Registering systemd factory Jun 03 16:17:16.170421 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.170381 2563 factory.go:223] Registration of the systemd container factory successfully Jun 03 16:17:16.170421 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.170399 2563 factory.go:103] Registering Raw factory Jun 03 16:17:16.170421 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.170416 2563 manager.go:1196] Started watching for new ooms in manager Jun 03 16:17:16.171696 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.171683 2563 manager.go:319] Starting recovery of all containers Jun 03 16:17:16.173669 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.173650 2563 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:16.177166 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.177101 2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-61.ec2.internal\" not found" node="ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.177166 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.177157 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-61.ec2.internal" not found Jun 03 16:17:16.183310 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.183147 2563 manager.go:324] Recovery completed Jun 03 16:17:16.187824 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.187809 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:16.190120 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.190103 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:16.190187 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.190134 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:16.190187 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.190148 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:16.190675 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.190662 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 16:17:16.190736 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.190674 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 16:17:16.190736 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.190697 2563 state_mem.go:36] "Initialized new in-memory state store" Jun 03 16:17:16.193981 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.193969 2563 policy_none.go:49] "None policy: Start" Jun 03 16:17:16.194026 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.193986 2563 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 16:17:16.194026 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.193997 2563 state_mem.go:35] "Initializing new in-memory state store" Jun 03 16:17:16.194895 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.194879 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-61.ec2.internal" not found Jun 03 16:17:16.232617 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.232598 2563 manager.go:341] "Starting Device Plugin manager" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.232631 2563 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.232641 2563 server.go:85] "Starting device plugin registration server" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.232874 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.232884 2563 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.232997 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.233072 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.233078 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.234063 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.234106 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.251673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.250618 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-61.ec2.internal" not found Jun 03 16:17:16.309963 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.309848 2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 16:17:16.311335 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.311319 2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 16:17:16.311429 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.311350 2563 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 16:17:16.311429 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.311373 2563 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 16:17:16.311429 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.311384 2563 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 16:17:16.311554 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.311433 2563 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 16:17:16.315166 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.315144 2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:16.333479 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.333451 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:16.334491 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.334477 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:16.334560 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.334512 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:16.334560 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.334523 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:16.334560 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.334548 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.340781 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.340766 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.340832 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.340791 2563 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-61.ec2.internal\": node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.364262 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.362445 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.411735 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.411663 2563 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal"] Jun 03 16:17:16.411816 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.411788 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:16.412793 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.412780 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:16.412849 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.412809 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:16.412849 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.412825 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:16.414192 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414181 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:16.414352 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414337 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.414405 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414367 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:16.414906 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414881 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:16.414906 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414905 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:16.415056 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414918 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:16.415056 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414933 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:16.415056 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414970 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:16.415056 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.414980 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:16.416181 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.416167 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.416247 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.416190 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:16.416828 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.416814 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:16.416904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.416843 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:16.416904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.416858 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:16.439542 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.439521 2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-61.ec2.internal\" not found" node="ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.442929 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.442913 2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-61.ec2.internal\" not found" node="ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.462540 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.462509 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.470995 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.470965 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/261ba64ea42df7bd667c10f964adbd79-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal\" (UID: \"261ba64ea42df7bd667c10f964adbd79\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.563266 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.563166 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.571423 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.571401 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34dbc6c38db1d2dc2bb23682a33284f5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-61.ec2.internal\" (UID: \"34dbc6c38db1d2dc2bb23682a33284f5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.571530 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.571511 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/261ba64ea42df7bd667c10f964adbd79-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal\" (UID: \"261ba64ea42df7bd667c10f964adbd79\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.571588 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.571552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/261ba64ea42df7bd667c10f964adbd79-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal\" (UID: \"261ba64ea42df7bd667c10f964adbd79\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.571645 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.571599 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/261ba64ea42df7bd667c10f964adbd79-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal\" (UID: \"261ba64ea42df7bd667c10f964adbd79\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.663913 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.663881 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.672204 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.672184 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/261ba64ea42df7bd667c10f964adbd79-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal\" (UID: \"261ba64ea42df7bd667c10f964adbd79\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.672263 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.672210 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34dbc6c38db1d2dc2bb23682a33284f5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-61.ec2.internal\" (UID: \"34dbc6c38db1d2dc2bb23682a33284f5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.672263 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.672235 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34dbc6c38db1d2dc2bb23682a33284f5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-61.ec2.internal\" (UID: \"34dbc6c38db1d2dc2bb23682a33284f5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.672329 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.672263 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/261ba64ea42df7bd667c10f964adbd79-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal\" (UID: \"261ba64ea42df7bd667c10f964adbd79\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.742330 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.742291 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.747652 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:16.747630 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" Jun 03 16:17:16.764991 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.764964 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.865547 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.865450 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:16.965967 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:16.965917 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:17.027699 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.027659 2563 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:17.060480 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.060454 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 16:17:17.060929 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.060600 2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 16:17:17.060929 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.060626 2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 16:17:17.060929 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.060625 2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 16:17:17.066587 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:17.066565 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:17.159560 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.159466 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 16:12:16 +0000 UTC" deadline="2027-12-30 23:05:13.654374561 +0000 UTC" Jun 03 16:17:17.159560 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.159504 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13806h47m56.494873484s" Jun 03 16:17:17.167647 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:17.167623 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:17.169811 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.169783 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 16:17:17.187648 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.187627 2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 16:17:17.213547 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.213523 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b7kff" Jun 03 16:17:17.219768 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.219750 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b7kff" Jun 03 16:17:17.268105 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:17.268073 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:17.346990 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:17.346929 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261ba64ea42df7bd667c10f964adbd79.slice/crio-e2031e36146667d821b88c8a3953f9b06fd583dc9b6f60d66d1190c4b7e80b87 WatchSource:0}: Error finding container e2031e36146667d821b88c8a3953f9b06fd583dc9b6f60d66d1190c4b7e80b87: Status 404 returned error can't find the container with id e2031e36146667d821b88c8a3953f9b06fd583dc9b6f60d66d1190c4b7e80b87 Jun 03 16:17:17.347521 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:17.347491 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34dbc6c38db1d2dc2bb23682a33284f5.slice/crio-65f4a117d9814a025b5108e444e4c19efc0cfeaff9221376d78625f9094f1dc6 WatchSource:0}: Error finding container 65f4a117d9814a025b5108e444e4c19efc0cfeaff9221376d78625f9094f1dc6: Status 404 returned error can't find the container with id 65f4a117d9814a025b5108e444e4c19efc0cfeaff9221376d78625f9094f1dc6 Jun 03 16:17:17.352338 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.352310 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:17:17.368596 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:17.368568 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-61.ec2.internal\" not found" Jun 03 16:17:17.373872 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.373855 2563 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:17.469774 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.469697 2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" Jun 03 16:17:17.481007 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.480990 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 16:17:17.481885 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.481873 2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" Jun 03 16:17:17.491006 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:17.490989 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 16:17:18.138048 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.138012 2563 apiserver.go:52] "Watching apiserver" Jun 03 16:17:18.143988 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.143932 2563 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 16:17:18.144419 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.144397 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-hcjql","openshift-ovn-kubernetes/ovnkube-node-htflp","openshift-cluster-node-tuning-operator/tuned-kmx2b","openshift-dns/node-resolver-g279r","openshift-image-registry/node-ca-mqcn6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal","openshift-multus/multus-additional-cni-plugins-mlqp5","openshift-multus/network-metrics-daemon-2xsw7","openshift-network-operator/iptables-alerter-sr7jk","kube-system/konnectivity-agent-sww98","kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8","openshift-multus/multus-9q2rw"] Jun 03 16:17:18.148903 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.148875 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.150773 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.150753 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.152041 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.151839 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 16:17:18.152971 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.152439 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 16:17:18.152971 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.152770 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.152971 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.152872 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 16:17:18.155089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.153538 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qtxqj\"" Jun 03 16:17:18.155089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.153973 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 16:17:18.155089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.154133 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 16:17:18.155089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.154280 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 16:17:18.155089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.154508 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 16:17:18.155089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.154750 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kgx7b\"" Jun 03 16:17:18.155089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.154916 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 16:17:18.155575 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.155122 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 16:17:18.155575 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.155201 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.155575 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.155450 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g7t87\"" Jun 03 16:17:18.155972 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.155801 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 16:17:18.155972 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.155936 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 16:17:18.156432 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.156066 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 16:17:18.158402 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.157191 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 16:17:18.158402 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.157208 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 16:17:18.158402 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.157313 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:18.158402 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.157388 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:18.158402 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.157790 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zjwzg\"" Jun 03 16:17:18.158402 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.157906 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 16:17:18.158402 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.158121 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 16:17:18.159574 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.159545 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.159879 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.159706 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:18.159879 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.159831 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:18.161278 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.161158 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.167688 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.165231 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7lkwj\"" Jun 03 16:17:18.167688 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.165273 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:17:18.167688 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.165823 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 16:17:18.167688 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.166215 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 16:17:18.167688 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.166252 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:17:18.167688 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.166522 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:18.167688 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.166688 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 16:17:18.167688 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.167123 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-txncn\"" Jun 03 16:17:18.168202 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.168139 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.168261 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.168244 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.168599 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.168567 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 16:17:18.169035 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.168875 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mx2mz\"" Jun 03 16:17:18.169119 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.169014 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 16:17:18.172424 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.172405 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bqxrb\"" Jun 03 16:17:18.173189 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.172645 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 16:17:18.173189 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.172659 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 16:17:18.173189 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.172879 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 16:17:18.173189 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.172907 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xvgj4\"" Jun 03 16:17:18.173394 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.173214 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 16:17:18.175890 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.175763 2563 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 16:17:18.180469 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180437 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.180565 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180475 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-cni-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.180565 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180501 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e16c3c6f-c435-43c3-a3cc-cd43071c7412-cni-binary-copy\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.180565 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180525 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-ovn\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.180565 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-run-ovn-kubernetes\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.180735 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysctl-conf\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.180735 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180596 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60c658c6-ac70-44dd-9644-69ed51bef4e1-konnectivity-ca\") pod \"konnectivity-agent-sww98\" (UID: \"60c658c6-ac70-44dd-9644-69ed51bef4e1\") " pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:18.180735 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-systemd\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.180735 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovnkube-script-lib\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.180735 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180657 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-tmp-dir\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.180735 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180679 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7cbl\" (UniqueName: \"kubernetes.io/projected/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-kube-api-access-z7cbl\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.180735 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180715 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42t6f\" (UniqueName: \"kubernetes.io/projected/d86094b1-a6e7-4c72-8291-ca83b6444a09-kube-api-access-42t6f\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-socket-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180820 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-cnibin\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180843 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-daemon-config\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-systemd-units\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-registration-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vgkl\" (UniqueName: \"kubernetes.io/projected/87401c41-2509-41ed-b469-c14c381c85c5-kube-api-access-4vgkl\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180959 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-cni-multus\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.180983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysctl-d\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-run\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-sys-fs\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.181046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181050 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhljg\" (UniqueName: \"kubernetes.io/projected/f754daa3-4c89-4f39-a9d3-91b61ce13bca-kube-api-access-zhljg\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-conf-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181172 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovnkube-config\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181196 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxwqc\" (UniqueName: \"kubernetes.io/projected/51336783-9c76-4705-8d2a-d9f3ad0e6089-kube-api-access-lxwqc\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181218 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-system-cni-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181241 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-env-overrides\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-kubernetes\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-lib-modules\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181329 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-modprobe-d\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181374 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-etc-kubernetes\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181405 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-os-release\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovn-node-metrics-cert\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181465 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-hosts-file\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181488 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs6kg\" (UniqueName: \"kubernetes.io/projected/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-kube-api-access-hs6kg\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181513 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdg6x\" (UniqueName: \"kubernetes.io/projected/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-kube-api-access-hdg6x\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181553 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-hostroot\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.181620 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181576 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-cni-netd\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181604 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181628 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d86094b1-a6e7-4c72-8291-ca83b6444a09-host-slash\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181650 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181674 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-device-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.181695 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-node-log\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182065 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-tuned\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv6dz\" (UniqueName: \"kubernetes.io/projected/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-kube-api-access-mv6dz\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182113 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cnibin\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182135 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zklks\" (UniqueName: \"kubernetes.io/projected/e16c3c6f-c435-43c3-a3cc-cd43071c7412-kube-api-access-zklks\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-run-netns\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182203 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysconfig\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182224 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-systemd\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182260 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-sys\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.182366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182295 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-var-lib-kubelet\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182321 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-tmp\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182348 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-kubelet\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-multus-certs\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182390 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-host\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-k8s-cni-cncf-io\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-etc-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182470 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-log-socket\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182495 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d86094b1-a6e7-4c72-8291-ca83b6444a09-iptables-alerter-script\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182514 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-cni-bin\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182539 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-slash\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182561 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-system-cni-dir\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182577 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-os-release\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182614 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-socket-dir-parent\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182662 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-host\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.182899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.183453 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182712 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cni-binary-copy\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.183453 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-cni-bin\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.183453 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-kubelet\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.183453 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-serviceca\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.183453 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182785 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60c658c6-ac70-44dd-9644-69ed51bef4e1-agent-certs\") pod \"konnectivity-agent-sww98\" (UID: \"60c658c6-ac70-44dd-9644-69ed51bef4e1\") " pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:18.183453 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182799 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-netns\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.183453 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.182821 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-var-lib-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.220512 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.220478 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:12:17 +0000 UTC" deadline="2028-03-13 11:28:31.775418606 +0000 UTC" Jun 03 16:17:18.220512 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.220505 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15571h11m13.554915572s" Jun 03 16:17:18.283061 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283029 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-slash\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.283231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283070 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-system-cni-dir\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.283231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283095 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-os-release\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.283231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283116 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.283231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283138 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-socket-dir-parent\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.283231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283151 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-system-cni-dir\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.283231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283151 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-slash\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.283231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283156 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-host\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.283231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283223 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-host\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283209 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-socket-dir-parent\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283249 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283264 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-os-release\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283279 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cni-binary-copy\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283308 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-cni-bin\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283333 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-kubelet\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283363 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-serviceca\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283373 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283387 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60c658c6-ac70-44dd-9644-69ed51bef4e1-agent-certs\") pod \"konnectivity-agent-sww98\" (UID: \"60c658c6-ac70-44dd-9644-69ed51bef4e1\") " pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283417 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-kubelet\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283412 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-netns\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283459 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-var-lib-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283467 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-netns\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.283506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283486 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283515 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-cni-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283541 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e16c3c6f-c435-43c3-a3cc-cd43071c7412-cni-binary-copy\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283564 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-ovn\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283588 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-run-ovn-kubernetes\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283610 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysctl-conf\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283636 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60c658c6-ac70-44dd-9644-69ed51bef4e1-konnectivity-ca\") pod \"konnectivity-agent-sww98\" (UID: \"60c658c6-ac70-44dd-9644-69ed51bef4e1\") " pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283659 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-systemd\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283684 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovnkube-script-lib\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283747 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-tmp-dir\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283777 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7cbl\" (UniqueName: \"kubernetes.io/projected/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-kube-api-access-z7cbl\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283805 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283809 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283844 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-42t6f\" (UniqueName: \"kubernetes.io/projected/d86094b1-a6e7-4c72-8291-ca83b6444a09-kube-api-access-42t6f\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283870 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-socket-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283877 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-run-ovn-kubernetes\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283887 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-serviceca\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.284092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283895 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-cnibin\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283918 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-cni-bin\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283922 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-daemon-config\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283967 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-systemd-units\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283868 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cni-binary-copy\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.283966 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-systemd\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284112 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysctl-conf\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284171 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-systemd-units\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284243 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284337 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.284596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284375 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-ovn\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284668 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-tmp-dir\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284731 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-var-lib-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284776 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-cnibin\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284820 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-registration-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284848 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4vgkl\" (UniqueName: \"kubernetes.io/projected/87401c41-2509-41ed-b469-c14c381c85c5-kube-api-access-4vgkl\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284876 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-cni-multus\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284903 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysctl-d\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284927 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-daemon-config\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.284984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284966 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-run\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.284990 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-sys-fs\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285012 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285033 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zhljg\" (UniqueName: \"kubernetes.io/projected/f754daa3-4c89-4f39-a9d3-91b61ce13bca-kube-api-access-zhljg\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285061 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-conf-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285060 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovnkube-script-lib\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285070 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysctl-d\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285184 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e16c3c6f-c435-43c3-a3cc-cd43071c7412-cni-binary-copy\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285190 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovnkube-config\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285198 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-registration-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285220 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lxwqc\" (UniqueName: \"kubernetes.io/projected/51336783-9c76-4705-8d2a-d9f3ad0e6089-kube-api-access-lxwqc\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285239 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-cni-multus\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285247 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-system-cni-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285278 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-run\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285300 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-env-overrides\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285308 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-sys-fs\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285388 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.285363 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285391 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60c658c6-ac70-44dd-9644-69ed51bef4e1-konnectivity-ca\") pod \"konnectivity-agent-sww98\" (UID: \"60c658c6-ac70-44dd-9644-69ed51bef4e1\") " pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285527 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-cni-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285547 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-socket-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285580 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-multus-conf-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285597 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-system-cni-dir\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285625 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-kubernetes\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285659 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-lib-modules\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285660 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-kubernetes\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285691 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-modprobe-d\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285732 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-etc-kubernetes\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285758 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-os-release\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285762 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-modprobe-d\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-lib-modules\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285781 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-env-overrides\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285796 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovn-node-metrics-cert\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285823 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-hosts-file\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285828 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-os-release\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285791 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-etc-kubernetes\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.286254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285847 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hs6kg\" (UniqueName: \"kubernetes.io/projected/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-kube-api-access-hs6kg\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285873 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hdg6x\" (UniqueName: \"kubernetes.io/projected/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-kube-api-access-hdg6x\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285885 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-hosts-file\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285983 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovnkube-config\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.285994 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-hostroot\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286024 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-cni-netd\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286053 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286037 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-hostroot\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286081 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d86094b1-a6e7-4c72-8291-ca83b6444a09-host-slash\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286105 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286110 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286128 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-device-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286142 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d86094b1-a6e7-4c72-8291-ca83b6444a09-host-slash\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286148 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286103 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-cni-netd\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286165 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286178 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-node-log\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286202 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-tuned\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286206 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-run-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286227 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-node-log\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286230 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mv6dz\" (UniqueName: \"kubernetes.io/projected/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-kube-api-access-mv6dz\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286205 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/87401c41-2509-41ed-b469-c14c381c85c5-device-dir\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286333 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cnibin\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286365 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286390 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zklks\" (UniqueName: \"kubernetes.io/projected/e16c3c6f-c435-43c3-a3cc-cd43071c7412-kube-api-access-zklks\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286425 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-run-netns\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286449 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysconfig\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286498 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-systemd\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286426 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f754daa3-4c89-4f39-a9d3-91b61ce13bca-cnibin\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286511 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-host-run-netns\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286521 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-sys\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286586 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-sys\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.286504 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286640 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-var-lib-kubelet\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.287874 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286644 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-sysconfig\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.286681 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs podName:0ac3b8bc-dd32-4bf5-97eb-84edbc66f369 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:18.786648171 +0000 UTC m=+3.081841211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs") pod "network-metrics-daemon-2xsw7" (UID: "0ac3b8bc-dd32-4bf5-97eb-84edbc66f369") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286674 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-systemd\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286700 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-var-lib-kubelet\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286732 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-tmp\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286756 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-kubelet\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286782 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-multus-certs\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286807 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-host\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286831 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-k8s-cni-cncf-io\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286837 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-kubelet\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286854 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-multus-certs\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286878 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-host\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286890 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-run-k8s-cni-cncf-io\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286895 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-etc-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286856 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-etc-openvswitch\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286926 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-log-socket\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286973 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d86094b1-a6e7-4c72-8291-ca83b6444a09-iptables-alerter-script\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.288511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286991 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51336783-9c76-4705-8d2a-d9f3ad0e6089-log-socket\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.289062 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.286999 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-cni-bin\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.289062 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.287055 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e16c3c6f-c435-43c3-a3cc-cd43071c7412-host-var-lib-cni-bin\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.289062 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.287473 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d86094b1-a6e7-4c72-8291-ca83b6444a09-iptables-alerter-script\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.289062 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.288659 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60c658c6-ac70-44dd-9644-69ed51bef4e1-agent-certs\") pod \"konnectivity-agent-sww98\" (UID: \"60c658c6-ac70-44dd-9644-69ed51bef4e1\") " pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:18.289062 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.289025 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51336783-9c76-4705-8d2a-d9f3ad0e6089-ovn-node-metrics-cert\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.290266 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.290192 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-etc-tuned\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.290700 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.290683 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-tmp\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.298141 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.297982 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:18.298141 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.298009 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:18.298141 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.298024 2563 projected.go:194] Error preparing data for projected volume kube-api-access-z7dhz for pod openshift-network-diagnostics/network-check-target-hcjql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:18.298141 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.298084 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz podName:9a72995b-271f-4ca9-b435-57c47fa87d13 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:18.798066025 +0000 UTC m=+3.093259070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z7dhz" (UniqueName: "kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz") pod "network-check-target-hcjql" (UID: "9a72995b-271f-4ca9-b435-57c47fa87d13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:18.301277 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.301251 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vgkl\" (UniqueName: \"kubernetes.io/projected/87401c41-2509-41ed-b469-c14c381c85c5-kube-api-access-4vgkl\") pod \"aws-ebs-csi-driver-node-dv5s8\" (UID: \"87401c41-2509-41ed-b469-c14c381c85c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.305157 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.305133 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zklks\" (UniqueName: \"kubernetes.io/projected/e16c3c6f-c435-43c3-a3cc-cd43071c7412-kube-api-access-zklks\") pod \"multus-9q2rw\" (UID: \"e16c3c6f-c435-43c3-a3cc-cd43071c7412\") " pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.305561 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.305538 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv6dz\" (UniqueName: \"kubernetes.io/projected/2c8d85be-78f4-41b1-8a06-770e8c5a95c1-kube-api-access-mv6dz\") pod \"tuned-kmx2b\" (UID: \"2c8d85be-78f4-41b1-8a06-770e8c5a95c1\") " pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.306039 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.305990 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs6kg\" (UniqueName: \"kubernetes.io/projected/175e42a6-fdf2-4684-91cb-53d33a7bc0fd-kube-api-access-hs6kg\") pod \"node-ca-mqcn6\" (UID: \"175e42a6-fdf2-4684-91cb-53d33a7bc0fd\") " pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.306130 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.306055 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42t6f\" (UniqueName: \"kubernetes.io/projected/d86094b1-a6e7-4c72-8291-ca83b6444a09-kube-api-access-42t6f\") pod \"iptables-alerter-sr7jk\" (UID: \"d86094b1-a6e7-4c72-8291-ca83b6444a09\") " pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.306196 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.306179 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhljg\" (UniqueName: \"kubernetes.io/projected/f754daa3-4c89-4f39-a9d3-91b61ce13bca-kube-api-access-zhljg\") pod \"multus-additional-cni-plugins-mlqp5\" (UID: \"f754daa3-4c89-4f39-a9d3-91b61ce13bca\") " pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.306404 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.306353 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxwqc\" (UniqueName: \"kubernetes.io/projected/51336783-9c76-4705-8d2a-d9f3ad0e6089-kube-api-access-lxwqc\") pod \"ovnkube-node-htflp\" (UID: \"51336783-9c76-4705-8d2a-d9f3ad0e6089\") " pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.306464 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.306445 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7cbl\" (UniqueName: \"kubernetes.io/projected/cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a-kube-api-access-z7cbl\") pod \"node-resolver-g279r\" (UID: \"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a\") " pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.308994 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.308975 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdg6x\" (UniqueName: \"kubernetes.io/projected/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-kube-api-access-hdg6x\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:18.316453 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.316375 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" event={"ID":"261ba64ea42df7bd667c10f964adbd79","Type":"ContainerStarted","Data":"e2031e36146667d821b88c8a3953f9b06fd583dc9b6f60d66d1190c4b7e80b87"} Jun 03 16:17:18.317399 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.317377 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" event={"ID":"34dbc6c38db1d2dc2bb23682a33284f5","Type":"ContainerStarted","Data":"65f4a117d9814a025b5108e444e4c19efc0cfeaff9221376d78625f9094f1dc6"} Jun 03 16:17:18.405221 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.405147 2563 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:18.469316 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.469276 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:18.487190 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.487162 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" Jun 03 16:17:18.495568 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.495545 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g279r" Jun 03 16:17:18.500095 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.500077 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mqcn6" Jun 03 16:17:18.508676 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.508658 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" Jun 03 16:17:18.521173 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.521149 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sr7jk" Jun 03 16:17:18.530700 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.530676 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:18.538235 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.538217 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" Jun 03 16:17:18.544789 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.544769 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9q2rw" Jun 03 16:17:18.633986 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.633935 2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:18.789625 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.789529 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:18.789798 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.789695 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:18.789798 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.789765 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs podName:0ac3b8bc-dd32-4bf5-97eb-84edbc66f369 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:19.78974527 +0000 UTC m=+4.084938302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs") pod "network-metrics-daemon-2xsw7" (UID: "0ac3b8bc-dd32-4bf5-97eb-84edbc66f369") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:18.890726 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:18.890687 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:18.890907 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.890825 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:18.890907 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.890844 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:18.890907 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.890856 2563 projected.go:194] Error preparing data for projected volume kube-api-access-z7dhz for pod openshift-network-diagnostics/network-check-target-hcjql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:18.891062 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:18.890911 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz podName:9a72995b-271f-4ca9-b435-57c47fa87d13 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:19.890891124 +0000 UTC m=+4.186084149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7dhz" (UniqueName: "kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz") pod "network-check-target-hcjql" (UID: "9a72995b-271f-4ca9-b435-57c47fa87d13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:19.137014 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.136982 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfefc2c0_1f78_4bec_b7cd_9b3e7a6fc09a.slice/crio-97854a5f44ef2500d4e868b1ba3ed0bc7093c559ea98b47dc3c14d6a28cafdaf WatchSource:0}: Error finding container 97854a5f44ef2500d4e868b1ba3ed0bc7093c559ea98b47dc3c14d6a28cafdaf: Status 404 returned error can't find the container with id 97854a5f44ef2500d4e868b1ba3ed0bc7093c559ea98b47dc3c14d6a28cafdaf Jun 03 16:17:19.150325 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.150301 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86094b1_a6e7_4c72_8291_ca83b6444a09.slice/crio-31cff0999220105052ca29d8e43815968937ac6e9254de00483da13addfb1b47 WatchSource:0}: Error finding container 31cff0999220105052ca29d8e43815968937ac6e9254de00483da13addfb1b47: Status 404 returned error can't find the container with id 31cff0999220105052ca29d8e43815968937ac6e9254de00483da13addfb1b47 Jun 03 16:17:19.154927 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.154845 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf754daa3_4c89_4f39_a9d3_91b61ce13bca.slice/crio-f02e1cc959fc3ea791b7c0a28f9cb58b05a829bd4522c799f5b0c253699af983 WatchSource:0}: Error finding container f02e1cc959fc3ea791b7c0a28f9cb58b05a829bd4522c799f5b0c253699af983: Status 404 returned error can't find the container with id f02e1cc959fc3ea791b7c0a28f9cb58b05a829bd4522c799f5b0c253699af983 Jun 03 16:17:19.156844 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.156815 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51336783_9c76_4705_8d2a_d9f3ad0e6089.slice/crio-cce71859e2b8891ca763eb25229fd3fa9ab69df6495e4b82e84063e930ad53ad WatchSource:0}: Error finding container cce71859e2b8891ca763eb25229fd3fa9ab69df6495e4b82e84063e930ad53ad: Status 404 returned error can't find the container with id cce71859e2b8891ca763eb25229fd3fa9ab69df6495e4b82e84063e930ad53ad Jun 03 16:17:19.157674 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.157652 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87401c41_2509_41ed_b469_c14c381c85c5.slice/crio-93c9872edce5bc4441a116a21ad03a340f9b6cb6fb45531c0a1037f8aa68069d WatchSource:0}: Error finding container 93c9872edce5bc4441a116a21ad03a340f9b6cb6fb45531c0a1037f8aa68069d: Status 404 returned error can't find the container with id 93c9872edce5bc4441a116a21ad03a340f9b6cb6fb45531c0a1037f8aa68069d Jun 03 16:17:19.158371 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.158286 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175e42a6_fdf2_4684_91cb_53d33a7bc0fd.slice/crio-e4f3ad6907835b90512721dec326b680048ece8fdce7c14bc1ce2187c7066174 WatchSource:0}: Error finding container e4f3ad6907835b90512721dec326b680048ece8fdce7c14bc1ce2187c7066174: Status 404 returned error can't find the container with id e4f3ad6907835b90512721dec326b680048ece8fdce7c14bc1ce2187c7066174 Jun 03 16:17:19.159577 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.159554 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8d85be_78f4_41b1_8a06_770e8c5a95c1.slice/crio-bd5e03e20dec25e23f710deb615fa6342bc4651e711fe40b9502936cbc11bdbc WatchSource:0}: Error finding container bd5e03e20dec25e23f710deb615fa6342bc4651e711fe40b9502936cbc11bdbc: Status 404 returned error can't find the container with id bd5e03e20dec25e23f710deb615fa6342bc4651e711fe40b9502936cbc11bdbc Jun 03 16:17:19.161392 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.161368 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c658c6_ac70_44dd_9644_69ed51bef4e1.slice/crio-ff1cd3cfd01ade882fd0a7ae97da8fc1568c01098388e2cf14810c7e666b7297 WatchSource:0}: Error finding container ff1cd3cfd01ade882fd0a7ae97da8fc1568c01098388e2cf14810c7e666b7297: Status 404 returned error can't find the container with id ff1cd3cfd01ade882fd0a7ae97da8fc1568c01098388e2cf14810c7e666b7297 Jun 03 16:17:19.161888 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:19.161855 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16c3c6f_c435_43c3_a3cc_cd43071c7412.slice/crio-68a7d1b74a2194b1194abb5d448c349120cb5e4f8890a286e7a8f30ce60e836e WatchSource:0}: Error finding container 68a7d1b74a2194b1194abb5d448c349120cb5e4f8890a286e7a8f30ce60e836e: Status 404 returned error can't find the container with id 68a7d1b74a2194b1194abb5d448c349120cb5e4f8890a286e7a8f30ce60e836e Jun 03 16:17:19.220982 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.220833 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:12:17 +0000 UTC" deadline="2028-02-24 18:48:58.18133644 +0000 UTC" Jun 03 16:17:19.220982 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.220983 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15146h31m38.960357965s" Jun 03 16:17:19.312408 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.312374 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:19.312561 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:19.312493 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:19.320370 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.320343 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mqcn6" event={"ID":"175e42a6-fdf2-4684-91cb-53d33a7bc0fd","Type":"ContainerStarted","Data":"e4f3ad6907835b90512721dec326b680048ece8fdce7c14bc1ce2187c7066174"} Jun 03 16:17:19.321436 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.321411 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"cce71859e2b8891ca763eb25229fd3fa9ab69df6495e4b82e84063e930ad53ad"} Jun 03 16:17:19.322328 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.322306 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g279r" event={"ID":"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a","Type":"ContainerStarted","Data":"97854a5f44ef2500d4e868b1ba3ed0bc7093c559ea98b47dc3c14d6a28cafdaf"} Jun 03 16:17:19.323867 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.323848 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" event={"ID":"34dbc6c38db1d2dc2bb23682a33284f5","Type":"ContainerStarted","Data":"0a59e3a8ad2651d2305ef0c447ee404460f06564943d8aaca89453d3b023d9e1"} Jun 03 16:17:19.324835 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.324812 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" event={"ID":"2c8d85be-78f4-41b1-8a06-770e8c5a95c1","Type":"ContainerStarted","Data":"bd5e03e20dec25e23f710deb615fa6342bc4651e711fe40b9502936cbc11bdbc"} Jun 03 16:17:19.325819 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.325802 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" event={"ID":"87401c41-2509-41ed-b469-c14c381c85c5","Type":"ContainerStarted","Data":"93c9872edce5bc4441a116a21ad03a340f9b6cb6fb45531c0a1037f8aa68069d"} Jun 03 16:17:19.326736 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.326703 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerStarted","Data":"f02e1cc959fc3ea791b7c0a28f9cb58b05a829bd4522c799f5b0c253699af983"} Jun 03 16:17:19.327781 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.327746 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sr7jk" event={"ID":"d86094b1-a6e7-4c72-8291-ca83b6444a09","Type":"ContainerStarted","Data":"31cff0999220105052ca29d8e43815968937ac6e9254de00483da13addfb1b47"} Jun 03 16:17:19.328835 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.328806 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9q2rw" event={"ID":"e16c3c6f-c435-43c3-a3cc-cd43071c7412","Type":"ContainerStarted","Data":"68a7d1b74a2194b1194abb5d448c349120cb5e4f8890a286e7a8f30ce60e836e"} Jun 03 16:17:19.329765 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.329738 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sww98" event={"ID":"60c658c6-ac70-44dd-9644-69ed51bef4e1","Type":"ContainerStarted","Data":"ff1cd3cfd01ade882fd0a7ae97da8fc1568c01098388e2cf14810c7e666b7297"} Jun 03 16:17:19.796757 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.796669 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:19.796922 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:19.796833 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:19.796922 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:19.796893 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs podName:0ac3b8bc-dd32-4bf5-97eb-84edbc66f369 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:21.796875978 +0000 UTC m=+6.092068997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs") pod "network-metrics-daemon-2xsw7" (UID: "0ac3b8bc-dd32-4bf5-97eb-84edbc66f369") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:19.897570 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.897501 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:19.897766 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:19.897681 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:19.897766 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:19.897708 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:19.897766 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:19.897721 2563 projected.go:194] Error preparing data for projected volume kube-api-access-z7dhz for pod openshift-network-diagnostics/network-check-target-hcjql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:19.897931 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:19.897783 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz podName:9a72995b-271f-4ca9-b435-57c47fa87d13 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:21.897761261 +0000 UTC m=+6.192954296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7dhz" (UniqueName: "kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz") pod "network-check-target-hcjql" (UID: "9a72995b-271f-4ca9-b435-57c47fa87d13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:19.974719 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:19.974308 2563 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:20.315985 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:20.314838 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:20.315985 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:20.314993 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:20.337121 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:20.334154 2563 generic.go:358] "Generic (PLEG): container finished" podID="261ba64ea42df7bd667c10f964adbd79" containerID="f85f87dc91b0138b74f454af6ec94ed31a91563cb557c5a227546050e6756456" exitCode=0 Jun 03 16:17:20.337121 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:20.334252 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" event={"ID":"261ba64ea42df7bd667c10f964adbd79","Type":"ContainerDied","Data":"f85f87dc91b0138b74f454af6ec94ed31a91563cb557c5a227546050e6756456"} Jun 03 16:17:20.351123 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:20.349692 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-61.ec2.internal" podStartSLOduration=3.349670784 podStartE2EDuration="3.349670784s" podCreationTimestamp="2026-06-03 16:17:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:17:19.335974323 +0000 UTC m=+3.631167363" watchObservedRunningTime="2026-06-03 16:17:20.349670784 +0000 UTC m=+4.644863825" Jun 03 16:17:21.313281 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:21.311928 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:21.313281 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:21.312081 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:21.355381 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:21.354594 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" event={"ID":"261ba64ea42df7bd667c10f964adbd79","Type":"ContainerStarted","Data":"60b1f1a66a5609ba9e1f31d71ea2ca66bbf72e2d8b39c78de4ad29e6ef42885e"} Jun 03 16:17:21.815528 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:21.815493 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:21.815713 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:21.815680 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:21.815820 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:21.815738 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs podName:0ac3b8bc-dd32-4bf5-97eb-84edbc66f369 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:25.81571741 +0000 UTC m=+10.110910457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs") pod "network-metrics-daemon-2xsw7" (UID: "0ac3b8bc-dd32-4bf5-97eb-84edbc66f369") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:21.916382 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:21.916343 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:21.916568 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:21.916497 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:21.916568 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:21.916518 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:21.916568 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:21.916530 2563 projected.go:194] Error preparing data for projected volume kube-api-access-z7dhz for pod openshift-network-diagnostics/network-check-target-hcjql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:21.916741 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:21.916590 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz podName:9a72995b-271f-4ca9-b435-57c47fa87d13 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:25.916562901 +0000 UTC m=+10.211755923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7dhz" (UniqueName: "kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz") pod "network-check-target-hcjql" (UID: "9a72995b-271f-4ca9-b435-57c47fa87d13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:22.312408 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:22.312318 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:22.312570 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:22.312487 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:23.312090 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:23.311576 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:23.312090 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:23.311717 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:24.312654 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:24.312612 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:24.313135 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:24.312729 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:25.311607 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:25.311572 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:25.311869 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:25.311703 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:25.851000 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:25.850959 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:25.851453 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:25.851128 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:25.851453 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:25.851199 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs podName:0ac3b8bc-dd32-4bf5-97eb-84edbc66f369 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:33.851179171 +0000 UTC m=+18.146372206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs") pod "network-metrics-daemon-2xsw7" (UID: "0ac3b8bc-dd32-4bf5-97eb-84edbc66f369") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:25.951908 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:25.951871 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:25.952116 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:25.952090 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:25.952116 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:25.952112 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:25.952224 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:25.952126 2563 projected.go:194] Error preparing data for projected volume kube-api-access-z7dhz for pod openshift-network-diagnostics/network-check-target-hcjql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:25.952224 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:25.952195 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz podName:9a72995b-271f-4ca9-b435-57c47fa87d13 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:33.952176247 +0000 UTC m=+18.247369269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7dhz" (UniqueName: "kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz") pod "network-check-target-hcjql" (UID: "9a72995b-271f-4ca9-b435-57c47fa87d13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:26.315402 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:26.313805 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:26.315402 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:26.313960 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:27.312072 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:27.312026 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:27.312503 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:27.312171 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:28.314067 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:28.314031 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:28.314519 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:28.314185 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:29.312087 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:29.312052 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:29.312250 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:29.312173 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:30.312609 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:30.312570 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:30.313079 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:30.312709 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:31.311860 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:31.311824 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:31.312053 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:31.311931 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:32.312803 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:32.312358 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:32.312803 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:32.312483 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:33.312246 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:33.312164 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:33.312397 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:33.312309 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:33.911391 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:33.911350 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:33.911827 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:33.911503 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:33.911827 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:33.911574 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs podName:0ac3b8bc-dd32-4bf5-97eb-84edbc66f369 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:49.911556716 +0000 UTC m=+34.206749739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs") pod "network-metrics-daemon-2xsw7" (UID: "0ac3b8bc-dd32-4bf5-97eb-84edbc66f369") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:34.012738 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:34.012697 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:34.012893 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:34.012841 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:34.012893 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:34.012858 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:34.012893 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:34.012868 2563 projected.go:194] Error preparing data for projected volume kube-api-access-z7dhz for pod openshift-network-diagnostics/network-check-target-hcjql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:34.013056 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:34.012917 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz podName:9a72995b-271f-4ca9-b435-57c47fa87d13 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:50.012903408 +0000 UTC m=+34.308096426 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7dhz" (UniqueName: "kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz") pod "network-check-target-hcjql" (UID: "9a72995b-271f-4ca9-b435-57c47fa87d13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:34.314271 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:34.314207 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:34.314400 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:34.314309 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:35.312304 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:35.312267 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:35.312750 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:35.312395 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:36.312738 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:36.312702 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:36.313232 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:36.312815 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:37.311964 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:37.311924 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:37.312087 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:37.312040 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:38.312412 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.311962 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:38.313175 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:38.312487 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:38.386991 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.386938 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g279r" event={"ID":"cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a","Type":"ContainerStarted","Data":"eaa9a5cd38d940a40ed9c127d1d695aedd381e9af189a60f510f6403e27389c2"} Jun 03 16:17:38.389109 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.389065 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" event={"ID":"2c8d85be-78f4-41b1-8a06-770e8c5a95c1","Type":"ContainerStarted","Data":"ca606e90f70d0c5d9927eb2a6f0b74e2521f89ce8533379de5454b7060f3a372"} Jun 03 16:17:38.390520 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.390494 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" event={"ID":"87401c41-2509-41ed-b469-c14c381c85c5","Type":"ContainerStarted","Data":"fa7fd9ad15f9167863fe0114d9608df0097565baad0122e37b5fe8e174a63ae0"} Jun 03 16:17:38.391981 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.391937 2563 generic.go:358] "Generic (PLEG): container finished" podID="f754daa3-4c89-4f39-a9d3-91b61ce13bca" containerID="d9899bc904eba6f716b91660c309979ba4ac01c86f56d6bcbdb904ad9ab468c1" exitCode=0 Jun 03 16:17:38.392093 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.392053 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerDied","Data":"d9899bc904eba6f716b91660c309979ba4ac01c86f56d6bcbdb904ad9ab468c1"} Jun 03 16:17:38.393673 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.393500 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9q2rw" event={"ID":"e16c3c6f-c435-43c3-a3cc-cd43071c7412","Type":"ContainerStarted","Data":"bcfb39737d73dc52dbf97a05ceee498f4fa6855a99a3ea423d8cf6171675b059"} Jun 03 16:17:38.394913 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.394878 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sww98" event={"ID":"60c658c6-ac70-44dd-9644-69ed51bef4e1","Type":"ContainerStarted","Data":"9edaab9b6ac0e98cb4d3f5fbe18c6878c439345c76202e09a91cec385ba6a741"} Jun 03 16:17:38.396048 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.396026 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mqcn6" event={"ID":"175e42a6-fdf2-4684-91cb-53d33a7bc0fd","Type":"ContainerStarted","Data":"ac08b7b1f7e6bc66c4d4083f4e73133d536e4ff3ddfbf8143d6d9749d9de4341"} Jun 03 16:17:38.398888 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.398865 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:17:38.399255 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.399217 2563 generic.go:358] "Generic (PLEG): container finished" podID="51336783-9c76-4705-8d2a-d9f3ad0e6089" containerID="cc2236e92b641e76a1b9240cde1e58fd9c9dfead037cb68b52830d098faec80c" exitCode=1 Jun 03 16:17:38.399255 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.399253 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"1c7111cd90e1b052015cdbd43a703fc913cd715a2e4169f3dc4926d23c048f4e"} Jun 03 16:17:38.399414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.399271 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"0c403d52f076f7e1e6e23b975ed99c74a58c7d8a375532623bbe1e1fabab482d"} Jun 03 16:17:38.399414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.399284 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"fd91f413f2731923e5e99656ccf559a8cd1d79b502b4f6c0ce6ec9b22bb7d7b7"} Jun 03 16:17:38.399414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.399297 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"07e340de4eb09964110637e93fad233fb37a37c4bf58c148c88538a47dab343c"} Jun 03 16:17:38.399414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.399311 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerDied","Data":"cc2236e92b641e76a1b9240cde1e58fd9c9dfead037cb68b52830d098faec80c"} Jun 03 16:17:38.399414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.399325 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"f7e624815b45d22e20039a3b76a06aa9d5fd66b3288741346e6d695e7da39397"} Jun 03 16:17:38.402885 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.402849 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-g279r" podStartSLOduration=4.200543429 podStartE2EDuration="22.402839485s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.148466821 +0000 UTC m=+3.443659840" lastFinishedPulling="2026-06-03 16:17:37.350762861 +0000 UTC m=+21.645955896" observedRunningTime="2026-06-03 16:17:38.40258808 +0000 UTC m=+22.697781121" watchObservedRunningTime="2026-06-03 16:17:38.402839485 +0000 UTC m=+22.698032526" Jun 03 16:17:38.402992 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.402936 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-61.ec2.internal" podStartSLOduration=21.402933384 podStartE2EDuration="21.402933384s" podCreationTimestamp="2026-06-03 16:17:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:17:21.369232805 +0000 UTC m=+5.664425847" watchObservedRunningTime="2026-06-03 16:17:38.402933384 +0000 UTC m=+22.698126428" Jun 03 16:17:38.418972 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.418916 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kmx2b" podStartSLOduration=4.212176111 podStartE2EDuration="22.418907022s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.16151319 +0000 UTC m=+3.456706222" lastFinishedPulling="2026-06-03 16:17:37.36824411 +0000 UTC m=+21.663437133" observedRunningTime="2026-06-03 16:17:38.418752128 +0000 UTC m=+22.713945168" watchObservedRunningTime="2026-06-03 16:17:38.418907022 +0000 UTC m=+22.714100063" Jun 03 16:17:38.441550 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.441507 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-9q2rw" podStartSLOduration=4.209087903 podStartE2EDuration="22.441491599s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.164018395 +0000 UTC m=+3.459211418" lastFinishedPulling="2026-06-03 16:17:37.396422082 +0000 UTC m=+21.691615114" observedRunningTime="2026-06-03 16:17:38.441069161 +0000 UTC m=+22.736262202" watchObservedRunningTime="2026-06-03 16:17:38.441491599 +0000 UTC m=+22.736684640" Jun 03 16:17:38.448909 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.448879 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 16:17:38.477522 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.477473 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mqcn6" podStartSLOduration=12.093846015 podStartE2EDuration="22.477458228s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.160494469 +0000 UTC m=+3.455687501" lastFinishedPulling="2026-06-03 16:17:29.544106681 +0000 UTC m=+13.839299714" observedRunningTime="2026-06-03 16:17:38.477448096 +0000 UTC m=+22.772641137" watchObservedRunningTime="2026-06-03 16:17:38.477458228 +0000 UTC m=+22.772651251" Jun 03 16:17:38.494932 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:38.494890 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sww98" podStartSLOduration=4.290626914 podStartE2EDuration="22.494878467s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.163013995 +0000 UTC m=+3.458207013" lastFinishedPulling="2026-06-03 16:17:37.367265534 +0000 UTC m=+21.662458566" observedRunningTime="2026-06-03 16:17:38.494713509 +0000 UTC m=+22.789906571" watchObservedRunningTime="2026-06-03 16:17:38.494878467 +0000 UTC m=+22.790071508" Jun 03 16:17:39.244345 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.244256 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T16:17:38.44890543Z","UUID":"c578b146-16d7-4008-b8df-c5c5aa8b45ae","Handler":null,"Name":"","Endpoint":""} Jun 03 16:17:39.245745 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.245726 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 16:17:39.245845 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.245752 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 16:17:39.312533 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.312504 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:39.312916 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:39.312629 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:39.403275 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.403237 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" event={"ID":"87401c41-2509-41ed-b469-c14c381c85c5","Type":"ContainerStarted","Data":"8df311e261e5d67c05ae496cbb5acd023a7d656938c3c776370b4f3839f1d8ac"} Jun 03 16:17:39.403454 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.403281 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" event={"ID":"87401c41-2509-41ed-b469-c14c381c85c5","Type":"ContainerStarted","Data":"0c37068740b9beb75a76fff27cccde7aca26abd5a1a3429a5456be6d59ed9b55"} Jun 03 16:17:39.404740 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.404706 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sr7jk" event={"ID":"d86094b1-a6e7-4c72-8291-ca83b6444a09","Type":"ContainerStarted","Data":"e2858f80d809d7efd292d4ee039f5e8ac0da90883642024a52efcaef8c3f8e72"} Jun 03 16:17:39.422997 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.422925 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dv5s8" podStartSLOduration=3.399141432 podStartE2EDuration="23.422905704s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.159529383 +0000 UTC m=+3.454722416" lastFinishedPulling="2026-06-03 16:17:39.183293656 +0000 UTC m=+23.478486688" observedRunningTime="2026-06-03 16:17:39.4227694 +0000 UTC m=+23.717962442" watchObservedRunningTime="2026-06-03 16:17:39.422905704 +0000 UTC m=+23.718098742" Jun 03 16:17:39.437080 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:39.437027 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sr7jk" podStartSLOduration=5.222864777 podStartE2EDuration="23.437011953s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.152696515 +0000 UTC m=+3.447889537" lastFinishedPulling="2026-06-03 16:17:37.366843679 +0000 UTC m=+21.662036713" observedRunningTime="2026-06-03 16:17:39.436464281 +0000 UTC m=+23.731657333" watchObservedRunningTime="2026-06-03 16:17:39.437011953 +0000 UTC m=+23.732204993" Jun 03 16:17:40.312058 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:40.311880 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:40.312210 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:40.312147 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:40.409603 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:40.409522 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:17:40.410059 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:40.409909 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"be81edd6496738573f96c97a232e8b67a3881cc2098104dd3baf729f4215b9f8"} Jun 03 16:17:41.312389 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:41.312358 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:41.312542 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:41.312477 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:41.368295 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:41.368261 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:41.368860 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:41.368828 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:41.412491 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:41.412460 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:41.413050 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:41.413033 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sww98" Jun 03 16:17:42.312485 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:42.312448 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:42.312663 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:42.312619 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:43.311705 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:43.311673 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:43.312283 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:43.311779 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:44.312250 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.312082 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:44.312782 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:44.312329 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:44.425374 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.425343 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:17:44.425672 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.425651 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"f660b0c333b3e32224418839cef46e4e6be3fe60d819a264026da2f2c7936b56"} Jun 03 16:17:44.425939 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.425920 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:44.426036 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.425963 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:44.426141 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.426126 2563 scope.go:117] "RemoveContainer" containerID="cc2236e92b641e76a1b9240cde1e58fd9c9dfead037cb68b52830d098faec80c" Jun 03 16:17:44.427416 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.427391 2563 generic.go:358] "Generic (PLEG): container finished" podID="f754daa3-4c89-4f39-a9d3-91b61ce13bca" containerID="333d377c344a906bcee27774242e0e3f4962941d02243cc5a462902e7ccc9a52" exitCode=0 Jun 03 16:17:44.427503 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.427426 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerDied","Data":"333d377c344a906bcee27774242e0e3f4962941d02243cc5a462902e7ccc9a52"} Jun 03 16:17:44.442667 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:44.442644 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:45.311671 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.311646 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:45.311807 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:45.311782 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:45.431893 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.431872 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:17:45.432503 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.432337 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" event={"ID":"51336783-9c76-4705-8d2a-d9f3ad0e6089","Type":"ContainerStarted","Data":"a474421bc5ebc1d4e3acb4843c628d05c25bc777a015d248be928a8478ed0e2f"} Jun 03 16:17:45.432596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.432576 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:45.434268 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.434238 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerStarted","Data":"5c51ee7ec54c286d52297fe31644797725f7a817f0e5087c4ed8d38807ceac9d"} Jun 03 16:17:45.449632 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.449611 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:17:45.477748 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.477708 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" podStartSLOduration=11.206987584 podStartE2EDuration="29.477693509s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.15869427 +0000 UTC m=+3.453887297" lastFinishedPulling="2026-06-03 16:17:37.429400189 +0000 UTC m=+21.724593222" observedRunningTime="2026-06-03 16:17:45.475160882 +0000 UTC m=+29.770353926" watchObservedRunningTime="2026-06-03 16:17:45.477693509 +0000 UTC m=+29.772886551" Jun 03 16:17:45.759394 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.759364 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2xsw7"] Jun 03 16:17:45.759555 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.759491 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:45.759644 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:45.759613 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:45.760217 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.760195 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hcjql"] Jun 03 16:17:45.760318 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:45.760290 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:45.760388 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:45.760367 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:46.439859 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:46.439819 2563 generic.go:358] "Generic (PLEG): container finished" podID="f754daa3-4c89-4f39-a9d3-91b61ce13bca" containerID="5c51ee7ec54c286d52297fe31644797725f7a817f0e5087c4ed8d38807ceac9d" exitCode=0 Jun 03 16:17:46.440514 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:46.440486 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerDied","Data":"5c51ee7ec54c286d52297fe31644797725f7a817f0e5087c4ed8d38807ceac9d"} Jun 03 16:17:47.311929 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:47.311898 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:47.312090 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:47.311900 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:47.312090 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:47.312025 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:47.312090 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:47.312072 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:48.445207 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:48.445176 2563 generic.go:358] "Generic (PLEG): container finished" podID="f754daa3-4c89-4f39-a9d3-91b61ce13bca" containerID="913000b5612cb8a1df5f82de9363ec3d89c4fea5a8b825916329d7aa82c7453f" exitCode=0 Jun 03 16:17:48.445799 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:48.445213 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerDied","Data":"913000b5612cb8a1df5f82de9363ec3d89c4fea5a8b825916329d7aa82c7453f"} Jun 03 16:17:49.312349 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.312277 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:49.312596 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.312289 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:49.312596 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:49.312394 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hcjql" podUID="9a72995b-271f-4ca9-b435-57c47fa87d13" Jun 03 16:17:49.312596 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:49.312466 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xsw7" podUID="0ac3b8bc-dd32-4bf5-97eb-84edbc66f369" Jun 03 16:17:49.569049 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.568965 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-61.ec2.internal" event="NodeReady" Jun 03 16:17:49.569508 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.569131 2563 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 16:17:49.620122 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.620088 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-czpwp"] Jun 03 16:17:49.640428 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.640399 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7rsgf"] Jun 03 16:17:49.640605 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.640585 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.644220 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.644067 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h928k\"" Jun 03 16:17:49.644220 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.644110 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 16:17:49.644220 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.644120 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 16:17:49.664574 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.664545 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-czpwp"] Jun 03 16:17:49.664741 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.664584 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7rsgf"] Jun 03 16:17:49.664741 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.664718 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:49.667293 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.667259 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 16:17:49.667293 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.667281 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 16:17:49.667477 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.667312 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 16:17:49.667568 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.667536 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nmtcn\"" Jun 03 16:17:49.724457 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.724418 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08e7d1e6-c8f1-421b-8210-138ca76e7354-config-volume\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.724457 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.724458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08e7d1e6-c8f1-421b-8210-138ca76e7354-tmp-dir\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.724690 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.724511 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68sc7\" (UniqueName: \"kubernetes.io/projected/08e7d1e6-c8f1-421b-8210-138ca76e7354-kube-api-access-68sc7\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.724690 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.724570 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.825150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.825073 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.825150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.825114 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08e7d1e6-c8f1-421b-8210-138ca76e7354-config-volume\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.825150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.825136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:49.825401 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.825158 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08e7d1e6-c8f1-421b-8210-138ca76e7354-tmp-dir\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.825401 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.825205 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68sc7\" (UniqueName: \"kubernetes.io/projected/08e7d1e6-c8f1-421b-8210-138ca76e7354-kube-api-access-68sc7\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.825401 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:49.825233 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:49.825401 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:49.825321 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls podName:08e7d1e6-c8f1-421b-8210-138ca76e7354 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:50.325298917 +0000 UTC m=+34.620491959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls") pod "dns-default-czpwp" (UID: "08e7d1e6-c8f1-421b-8210-138ca76e7354") : secret "dns-default-metrics-tls" not found Jun 03 16:17:49.825568 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.825533 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08e7d1e6-c8f1-421b-8210-138ca76e7354-tmp-dir\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.825568 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.825233 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4ghs\" (UniqueName: \"kubernetes.io/projected/deec8169-7f63-4f6d-bcfb-f35deee23591-kube-api-access-l4ghs\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:49.825742 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.825720 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08e7d1e6-c8f1-421b-8210-138ca76e7354-config-volume\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.835634 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.835612 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68sc7\" (UniqueName: \"kubernetes.io/projected/08e7d1e6-c8f1-421b-8210-138ca76e7354-kube-api-access-68sc7\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:49.926817 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.926775 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l4ghs\" (UniqueName: \"kubernetes.io/projected/deec8169-7f63-4f6d-bcfb-f35deee23591-kube-api-access-l4ghs\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:49.927028 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.926836 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:49.927028 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.926891 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:49.927028 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:49.926961 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:49.927028 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:49.927002 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:49.927028 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:49.927026 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs podName:0ac3b8bc-dd32-4bf5-97eb-84edbc66f369 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:21.927006564 +0000 UTC m=+66.222199595 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs") pod "network-metrics-daemon-2xsw7" (UID: "0ac3b8bc-dd32-4bf5-97eb-84edbc66f369") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:49.927295 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:49.927044 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert podName:deec8169-7f63-4f6d-bcfb-f35deee23591 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:50.427034224 +0000 UTC m=+34.722227244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert") pod "ingress-canary-7rsgf" (UID: "deec8169-7f63-4f6d-bcfb-f35deee23591") : secret "canary-serving-cert" not found Jun 03 16:17:49.935492 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:49.935461 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4ghs\" (UniqueName: \"kubernetes.io/projected/deec8169-7f63-4f6d-bcfb-f35deee23591-kube-api-access-l4ghs\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:50.027618 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.027580 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:50.027777 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:50.027727 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:50.027777 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:50.027744 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:50.027777 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:50.027753 2563 projected.go:194] Error preparing data for projected volume kube-api-access-z7dhz for pod openshift-network-diagnostics/network-check-target-hcjql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:50.027887 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:50.027798 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz podName:9a72995b-271f-4ca9-b435-57c47fa87d13 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:22.027785246 +0000 UTC m=+66.322978265 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7dhz" (UniqueName: "kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz") pod "network-check-target-hcjql" (UID: "9a72995b-271f-4ca9-b435-57c47fa87d13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:50.066623 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.066596 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g279r_cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a/dns-node-resolver/0.log" Jun 03 16:17:50.330221 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.330185 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:50.330410 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:50.330350 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:50.330470 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:50.330428 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls podName:08e7d1e6-c8f1-421b-8210-138ca76e7354 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:51.330409506 +0000 UTC m=+35.625602542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls") pod "dns-default-czpwp" (UID: "08e7d1e6-c8f1-421b-8210-138ca76e7354") : secret "dns-default-metrics-tls" not found Jun 03 16:17:50.351245 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.351216 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-xddr6"] Jun 03 16:17:50.380290 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.380261 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-xddr6"] Jun 03 16:17:50.380506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.380380 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.383150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.383120 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 03 16:17:50.383271 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.383167 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 03 16:17:50.383271 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.383125 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 03 16:17:50.383505 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.383487 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-s5x64\"" Jun 03 16:17:50.383767 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.383742 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 03 16:17:50.430723 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.430650 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:50.430906 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:50.430796 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:50.430906 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:50.430871 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert podName:deec8169-7f63-4f6d-bcfb-f35deee23591 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:51.43085227 +0000 UTC m=+35.726045295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert") pod "ingress-canary-7rsgf" (UID: "deec8169-7f63-4f6d-bcfb-f35deee23591") : secret "canary-serving-cert" not found Jun 03 16:17:50.531514 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.531473 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08667098-a56c-4a78-950c-690d20aed850-signing-cabundle\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.531703 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.531529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08667098-a56c-4a78-950c-690d20aed850-signing-key\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.531703 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.531661 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbgcl\" (UniqueName: \"kubernetes.io/projected/08667098-a56c-4a78-950c-690d20aed850-kube-api-access-tbgcl\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.632524 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.632487 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08667098-a56c-4a78-950c-690d20aed850-signing-cabundle\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.633116 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.632543 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08667098-a56c-4a78-950c-690d20aed850-signing-key\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.633116 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.632783 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tbgcl\" (UniqueName: \"kubernetes.io/projected/08667098-a56c-4a78-950c-690d20aed850-kube-api-access-tbgcl\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.633415 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.633384 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08667098-a56c-4a78-950c-690d20aed850-signing-cabundle\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.635172 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.635152 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08667098-a56c-4a78-950c-690d20aed850-signing-key\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.641600 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.641568 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbgcl\" (UniqueName: \"kubernetes.io/projected/08667098-a56c-4a78-950c-690d20aed850-kube-api-access-tbgcl\") pod \"service-ca-685df94d57-xddr6\" (UID: \"08667098-a56c-4a78-950c-690d20aed850\") " pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.691483 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.691447 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-xddr6" Jun 03 16:17:50.860484 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:50.860452 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-xddr6"] Jun 03 16:17:50.866246 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:17:50.866213 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08667098_a56c_4a78_950c_690d20aed850.slice/crio-1de927f2f3a8c832be054eb50b967b7933f2114c5d76fb5eb8eeecb28fdd857d WatchSource:0}: Error finding container 1de927f2f3a8c832be054eb50b967b7933f2114c5d76fb5eb8eeecb28fdd857d: Status 404 returned error can't find the container with id 1de927f2f3a8c832be054eb50b967b7933f2114c5d76fb5eb8eeecb28fdd857d Jun 03 16:17:51.251337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.251309 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mqcn6_175e42a6-fdf2-4684-91cb-53d33a7bc0fd/node-ca/0.log" Jun 03 16:17:51.311753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.311720 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:17:51.311960 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.311720 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:17:51.317690 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.317664 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 16:17:51.317909 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.317888 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 16:17:51.318128 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.318115 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 16:17:51.318570 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.318554 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cchx6\"" Jun 03 16:17:51.325159 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.325140 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f9ccm\"" Jun 03 16:17:51.337421 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.337395 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:51.337547 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:51.337531 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:51.337597 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:51.337589 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls podName:08e7d1e6-c8f1-421b-8210-138ca76e7354 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:53.337573393 +0000 UTC m=+37.632766415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls") pod "dns-default-czpwp" (UID: "08e7d1e6-c8f1-421b-8210-138ca76e7354") : secret "dns-default-metrics-tls" not found Jun 03 16:17:51.438558 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.438523 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:51.438720 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:51.438653 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:51.438779 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:51.438730 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert podName:deec8169-7f63-4f6d-bcfb-f35deee23591 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:53.438711126 +0000 UTC m=+37.733904148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert") pod "ingress-canary-7rsgf" (UID: "deec8169-7f63-4f6d-bcfb-f35deee23591") : secret "canary-serving-cert" not found Jun 03 16:17:51.452794 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:51.452751 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-xddr6" event={"ID":"08667098-a56c-4a78-950c-690d20aed850","Type":"ContainerStarted","Data":"1de927f2f3a8c832be054eb50b967b7933f2114c5d76fb5eb8eeecb28fdd857d"} Jun 03 16:17:53.354535 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:53.354497 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:53.355031 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:53.354675 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:53.355031 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:53.354751 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls podName:08e7d1e6-c8f1-421b-8210-138ca76e7354 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:57.354731881 +0000 UTC m=+41.649924916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls") pod "dns-default-czpwp" (UID: "08e7d1e6-c8f1-421b-8210-138ca76e7354") : secret "dns-default-metrics-tls" not found Jun 03 16:17:53.455053 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:53.455015 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:53.455231 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:53.455125 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:53.455231 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:53.455184 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert podName:deec8169-7f63-4f6d-bcfb-f35deee23591 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:57.455160631 +0000 UTC m=+41.750353650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert") pod "ingress-canary-7rsgf" (UID: "deec8169-7f63-4f6d-bcfb-f35deee23591") : secret "canary-serving-cert" not found Jun 03 16:17:55.462023 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:55.461985 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-xddr6" event={"ID":"08667098-a56c-4a78-950c-690d20aed850","Type":"ContainerStarted","Data":"fb170166c399ceabdb51f41a39fd8836f8116147a3ef4b68e55bea5bd763a7eb"} Jun 03 16:17:55.465130 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:55.465103 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerStarted","Data":"62896b131c3ce0f5cd1888a0ff23f6671ef2aa9cc385dd93b64a7c4cc573f6ab"} Jun 03 16:17:56.470606 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:56.470572 2563 generic.go:358] "Generic (PLEG): container finished" podID="f754daa3-4c89-4f39-a9d3-91b61ce13bca" containerID="62896b131c3ce0f5cd1888a0ff23f6671ef2aa9cc385dd93b64a7c4cc573f6ab" exitCode=0 Jun 03 16:17:56.471680 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:56.471657 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerDied","Data":"62896b131c3ce0f5cd1888a0ff23f6671ef2aa9cc385dd93b64a7c4cc573f6ab"} Jun 03 16:17:56.487142 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:56.487088 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-xddr6" podStartSLOduration=2.033710402 podStartE2EDuration="6.487074366s" podCreationTimestamp="2026-06-03 16:17:50 +0000 UTC" firstStartedPulling="2026-06-03 16:17:50.868664889 +0000 UTC m=+35.163857921" lastFinishedPulling="2026-06-03 16:17:55.322028863 +0000 UTC m=+39.617221885" observedRunningTime="2026-06-03 16:17:56.486129898 +0000 UTC m=+40.781322940" watchObservedRunningTime="2026-06-03 16:17:56.487074366 +0000 UTC m=+40.782267409" Jun 03 16:17:57.382804 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:57.382750 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:17:57.383016 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:57.382904 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:57.383016 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:57.382998 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls podName:08e7d1e6-c8f1-421b-8210-138ca76e7354 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:05.382978958 +0000 UTC m=+49.678171980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls") pod "dns-default-czpwp" (UID: "08e7d1e6-c8f1-421b-8210-138ca76e7354") : secret "dns-default-metrics-tls" not found Jun 03 16:17:57.474811 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:57.474782 2563 generic.go:358] "Generic (PLEG): container finished" podID="f754daa3-4c89-4f39-a9d3-91b61ce13bca" containerID="23463253d68dc1b5d06717f9c0e8495c213730ede5e90ee289bc7cb8a9988c7f" exitCode=0 Jun 03 16:17:57.475187 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:57.474837 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerDied","Data":"23463253d68dc1b5d06717f9c0e8495c213730ede5e90ee289bc7cb8a9988c7f"} Jun 03 16:17:57.483992 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:57.483941 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:17:57.484110 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:57.484048 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:57.484110 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:17:57.484101 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert podName:deec8169-7f63-4f6d-bcfb-f35deee23591 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:05.484087167 +0000 UTC m=+49.779280186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert") pod "ingress-canary-7rsgf" (UID: "deec8169-7f63-4f6d-bcfb-f35deee23591") : secret "canary-serving-cert" not found Jun 03 16:17:58.479744 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:58.479705 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" event={"ID":"f754daa3-4c89-4f39-a9d3-91b61ce13bca","Type":"ContainerStarted","Data":"eed4a140faa2b42578c250bc1a52b34788be907a360972bfcbda20172153d499"} Jun 03 16:17:58.503593 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:17:58.503533 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mlqp5" podStartSLOduration=6.343119536 podStartE2EDuration="42.503516685s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:19.156594041 +0000 UTC m=+3.451787063" lastFinishedPulling="2026-06-03 16:17:55.316991178 +0000 UTC m=+39.612184212" observedRunningTime="2026-06-03 16:17:58.502616643 +0000 UTC m=+42.797809678" watchObservedRunningTime="2026-06-03 16:17:58.503516685 +0000 UTC m=+42.798709727" Jun 03 16:18:05.441863 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:05.441827 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:18:05.445712 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:05.445676 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08e7d1e6-c8f1-421b-8210-138ca76e7354-metrics-tls\") pod \"dns-default-czpwp\" (UID: \"08e7d1e6-c8f1-421b-8210-138ca76e7354\") " pod="openshift-dns/dns-default-czpwp" Jun 03 16:18:05.542764 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:05.542725 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:18:05.545306 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:05.545282 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deec8169-7f63-4f6d-bcfb-f35deee23591-cert\") pod \"ingress-canary-7rsgf\" (UID: \"deec8169-7f63-4f6d-bcfb-f35deee23591\") " pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:18:05.552084 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:05.552061 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-czpwp" Jun 03 16:18:05.575914 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:05.575882 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7rsgf" Jun 03 16:18:05.720588 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:05.720515 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-czpwp"] Jun 03 16:18:05.723862 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:05.723827 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e7d1e6_c8f1_421b_8210_138ca76e7354.slice/crio-03bca6b50863d70bfd49dbb06f6d994e748543612307c5919322192bc6669420 WatchSource:0}: Error finding container 03bca6b50863d70bfd49dbb06f6d994e748543612307c5919322192bc6669420: Status 404 returned error can't find the container with id 03bca6b50863d70bfd49dbb06f6d994e748543612307c5919322192bc6669420 Jun 03 16:18:05.733274 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:05.733246 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7rsgf"] Jun 03 16:18:05.738587 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:05.738558 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeec8169_7f63_4f6d_bcfb_f35deee23591.slice/crio-4099b1d9e6b8a1ccd11047c3ffff15eb30faeecaaaa5a948bcfcf5da875ed95a WatchSource:0}: Error finding container 4099b1d9e6b8a1ccd11047c3ffff15eb30faeecaaaa5a948bcfcf5da875ed95a: Status 404 returned error can't find the container with id 4099b1d9e6b8a1ccd11047c3ffff15eb30faeecaaaa5a948bcfcf5da875ed95a Jun 03 16:18:06.497744 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:06.497700 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7rsgf" event={"ID":"deec8169-7f63-4f6d-bcfb-f35deee23591","Type":"ContainerStarted","Data":"4099b1d9e6b8a1ccd11047c3ffff15eb30faeecaaaa5a948bcfcf5da875ed95a"} Jun 03 16:18:06.498897 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:06.498868 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czpwp" event={"ID":"08e7d1e6-c8f1-421b-8210-138ca76e7354","Type":"ContainerStarted","Data":"03bca6b50863d70bfd49dbb06f6d994e748543612307c5919322192bc6669420"} Jun 03 16:18:08.505477 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:08.505433 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czpwp" event={"ID":"08e7d1e6-c8f1-421b-8210-138ca76e7354","Type":"ContainerStarted","Data":"ea61e153ec84ce8c4ab6cdb2596dcad739d7d23b071601fb8c79f8c49ad4a32d"} Jun 03 16:18:08.505477 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:08.505477 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czpwp" event={"ID":"08e7d1e6-c8f1-421b-8210-138ca76e7354","Type":"ContainerStarted","Data":"fabbf616b8005bd8021ddbf6b137e4acb7ed44b119592305969eeefe661e69c3"} Jun 03 16:18:08.505925 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:08.505553 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-czpwp" Jun 03 16:18:08.506763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:08.506742 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7rsgf" event={"ID":"deec8169-7f63-4f6d-bcfb-f35deee23591","Type":"ContainerStarted","Data":"6314df0236a194a6812ee2185a8f826dcb2379b9c3bf2ddc5f6b91a9e5da5ea7"} Jun 03 16:18:08.522134 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:08.522086 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-czpwp" podStartSLOduration=17.125757378 podStartE2EDuration="19.522072757s" podCreationTimestamp="2026-06-03 16:17:49 +0000 UTC" firstStartedPulling="2026-06-03 16:18:05.725703415 +0000 UTC m=+50.020896434" lastFinishedPulling="2026-06-03 16:18:08.12201859 +0000 UTC m=+52.417211813" observedRunningTime="2026-06-03 16:18:08.521511195 +0000 UTC m=+52.816704237" watchObservedRunningTime="2026-06-03 16:18:08.522072757 +0000 UTC m=+52.817265798" Jun 03 16:18:08.537026 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:08.536975 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7rsgf" podStartSLOduration=17.150809929 podStartE2EDuration="19.536936792s" podCreationTimestamp="2026-06-03 16:17:49 +0000 UTC" firstStartedPulling="2026-06-03 16:18:05.740522166 +0000 UTC m=+50.035715185" lastFinishedPulling="2026-06-03 16:18:08.126649024 +0000 UTC m=+52.421842048" observedRunningTime="2026-06-03 16:18:08.536282585 +0000 UTC m=+52.831475628" watchObservedRunningTime="2026-06-03 16:18:08.536936792 +0000 UTC m=+52.832129836" Jun 03 16:18:13.282080 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.282048 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-shkcj"] Jun 03 16:18:13.316730 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.316700 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-shkcj"] Jun 03 16:18:13.316875 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.316821 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-shkcj" Jun 03 16:18:13.320763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.320738 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 03 16:18:13.320889 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.320790 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 03 16:18:13.320889 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.320811 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ddtx9\"" Jun 03 16:18:13.379451 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.379412 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d4f7b9487-mb7cs"] Jun 03 16:18:13.399734 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.399707 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mn9rf"] Jun 03 16:18:13.399914 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.399891 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.402564 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.402539 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 03 16:18:13.403132 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.403111 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9hn4z\"" Jun 03 16:18:13.403496 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.403480 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 03 16:18:13.403808 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.403648 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 03 16:18:13.409647 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.409624 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 03 16:18:13.414384 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.414364 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mn9rf"] Jun 03 16:18:13.414384 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.414387 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d4f7b9487-mb7cs"] Jun 03 16:18:13.414506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.414475 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.416978 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.416935 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 16:18:13.417072 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.416998 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 16:18:13.417072 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.417021 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hq6wm\"" Jun 03 16:18:13.417072 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.417043 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 16:18:13.417193 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.417001 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 16:18:13.502377 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-image-registry-private-configuration\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.502377 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502378 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-registry-tls\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.502586 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk9p9\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-kube-api-access-rk9p9\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.502586 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502415 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-installation-pull-secrets\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.502586 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-registry-certificates\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.502586 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502471 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-bound-sa-token\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.502586 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-trusted-ca\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.502586 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502548 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-ca-trust-extracted\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.502586 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.502585 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98qh2\" (UniqueName: \"kubernetes.io/projected/f534aa01-28bb-4e35-83e5-9494f96eb8de-kube-api-access-98qh2\") pod \"downloads-c65b8dc4b-shkcj\" (UID: \"f534aa01-28bb-4e35-83e5-9494f96eb8de\") " pod="openshift-console/downloads-c65b8dc4b-shkcj" Jun 03 16:18:13.603445 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603361 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.603445 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603425 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-98qh2\" (UniqueName: \"kubernetes.io/projected/f534aa01-28bb-4e35-83e5-9494f96eb8de-kube-api-access-98qh2\") pod \"downloads-c65b8dc4b-shkcj\" (UID: \"f534aa01-28bb-4e35-83e5-9494f96eb8de\") " pod="openshift-console/downloads-c65b8dc4b-shkcj" Jun 03 16:18:13.603638 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603453 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-image-registry-private-configuration\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.603638 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603474 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-registry-tls\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.603638 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603497 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rk9p9\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-kube-api-access-rk9p9\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.603638 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603522 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-installation-pull-secrets\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.603638 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603542 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-registry-certificates\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.603638 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603563 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-bound-sa-token\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.603638 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-data-volume\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.604150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603769 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-trusted-ca\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.604150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-crio-socket\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.604150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603861 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-ca-trust-extracted\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.604150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scg9m\" (UniqueName: \"kubernetes.io/projected/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-kube-api-access-scg9m\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.604150 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.603928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.604545 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.604523 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-ca-trust-extracted\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.604638 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.604616 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-registry-certificates\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.604884 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.604868 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-trusted-ca\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.606180 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.606157 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-registry-tls\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.606546 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.606523 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-image-registry-private-configuration\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.614599 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.614568 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98qh2\" (UniqueName: \"kubernetes.io/projected/f534aa01-28bb-4e35-83e5-9494f96eb8de-kube-api-access-98qh2\") pod \"downloads-c65b8dc4b-shkcj\" (UID: \"f534aa01-28bb-4e35-83e5-9494f96eb8de\") " pod="openshift-console/downloads-c65b8dc4b-shkcj" Jun 03 16:18:13.616337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.616309 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-installation-pull-secrets\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.618224 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.618201 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk9p9\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-kube-api-access-rk9p9\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.618324 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.618266 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c19a3f9-7c3a-4eb1-be3b-d6595206f370-bound-sa-token\") pod \"image-registry-d4f7b9487-mb7cs\" (UID: \"7c19a3f9-7c3a-4eb1-be3b-d6595206f370\") " pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.625099 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.625081 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-shkcj" Jun 03 16:18:13.705086 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.705050 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-data-volume\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.705233 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.705134 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-crio-socket\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.705233 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.705167 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-scg9m\" (UniqueName: \"kubernetes.io/projected/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-kube-api-access-scg9m\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.705233 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.705198 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.705233 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.705232 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.705430 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.705360 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-crio-socket\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.705488 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.705438 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-data-volume\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.705790 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.705768 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.707658 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.707635 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.712321 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.712304 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:13.712564 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.712544 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scg9m\" (UniqueName: \"kubernetes.io/projected/e4f8d59e-c5ad-4920-8316-f8d4451d1ec1-kube-api-access-scg9m\") pod \"insights-runtime-extractor-mn9rf\" (UID: \"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1\") " pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.721889 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.721865 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mn9rf" Jun 03 16:18:13.758419 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.758361 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-shkcj"] Jun 03 16:18:13.765586 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:13.765536 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf534aa01_28bb_4e35_83e5_9494f96eb8de.slice/crio-c44720504a4f7f8165563774e81aca2dcdb20dbb717cb741d55ecac868200d81 WatchSource:0}: Error finding container c44720504a4f7f8165563774e81aca2dcdb20dbb717cb741d55ecac868200d81: Status 404 returned error can't find the container with id c44720504a4f7f8165563774e81aca2dcdb20dbb717cb741d55ecac868200d81 Jun 03 16:18:13.846414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.846382 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d4f7b9487-mb7cs"] Jun 03 16:18:13.850754 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:13.850726 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c19a3f9_7c3a_4eb1_be3b_d6595206f370.slice/crio-29567c883994748a8881c1f28a2b441e8903d5a059129ff94e3b6ba23af2b4ef WatchSource:0}: Error finding container 29567c883994748a8881c1f28a2b441e8903d5a059129ff94e3b6ba23af2b4ef: Status 404 returned error can't find the container with id 29567c883994748a8881c1f28a2b441e8903d5a059129ff94e3b6ba23af2b4ef Jun 03 16:18:13.864751 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:13.864685 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mn9rf"] Jun 03 16:18:13.868680 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:13.868657 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f8d59e_c5ad_4920_8316_f8d4451d1ec1.slice/crio-5381c4afa0448e68485651e8fdd9ec61b3fd2dc1c6373c9c6ec4cc20f1bc48e5 WatchSource:0}: Error finding container 5381c4afa0448e68485651e8fdd9ec61b3fd2dc1c6373c9c6ec4cc20f1bc48e5: Status 404 returned error can't find the container with id 5381c4afa0448e68485651e8fdd9ec61b3fd2dc1c6373c9c6ec4cc20f1bc48e5 Jun 03 16:18:14.525077 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:14.525035 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" event={"ID":"7c19a3f9-7c3a-4eb1-be3b-d6595206f370","Type":"ContainerStarted","Data":"dcdc8936b951b32c3af5217eec58034a68cc1ba4a6476840a7ee45b87b4ab0df"} Jun 03 16:18:14.525077 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:14.525081 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" event={"ID":"7c19a3f9-7c3a-4eb1-be3b-d6595206f370","Type":"ContainerStarted","Data":"29567c883994748a8881c1f28a2b441e8903d5a059129ff94e3b6ba23af2b4ef"} Jun 03 16:18:14.525531 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:14.525121 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:14.527466 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:14.527394 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-shkcj" event={"ID":"f534aa01-28bb-4e35-83e5-9494f96eb8de","Type":"ContainerStarted","Data":"c44720504a4f7f8165563774e81aca2dcdb20dbb717cb741d55ecac868200d81"} Jun 03 16:18:14.528895 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:14.528872 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mn9rf" event={"ID":"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1","Type":"ContainerStarted","Data":"2ecdf7782daf96777efb1d69b117c6103589084981d701f334bcf425ad8549f3"} Jun 03 16:18:14.529026 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:14.528901 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mn9rf" event={"ID":"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1","Type":"ContainerStarted","Data":"5381c4afa0448e68485651e8fdd9ec61b3fd2dc1c6373c9c6ec4cc20f1bc48e5"} Jun 03 16:18:14.545781 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:14.545726 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" podStartSLOduration=1.5457081559999999 podStartE2EDuration="1.545708156s" podCreationTimestamp="2026-06-03 16:18:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:18:14.544617587 +0000 UTC m=+58.839810628" watchObservedRunningTime="2026-06-03 16:18:14.545708156 +0000 UTC m=+58.840901201" Jun 03 16:18:15.534111 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:15.534069 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mn9rf" event={"ID":"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1","Type":"ContainerStarted","Data":"1b3f94ed7ce0ce8ca619f8a7c383b2400ebd483d93d22b0f6652e216bb61361d"} Jun 03 16:18:17.462965 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:17.462918 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htflp" Jun 03 16:18:17.541362 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:17.541327 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mn9rf" event={"ID":"e4f8d59e-c5ad-4920-8316-f8d4451d1ec1","Type":"ContainerStarted","Data":"d56e4931954dcee1f8d900a0d759776be364750031c6968808c59642e8e8122e"} Jun 03 16:18:17.563024 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:17.562966 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mn9rf" podStartSLOduration=1.435396536 podStartE2EDuration="4.562931495s" podCreationTimestamp="2026-06-03 16:18:13 +0000 UTC" firstStartedPulling="2026-06-03 16:18:13.973658482 +0000 UTC m=+58.268851504" lastFinishedPulling="2026-06-03 16:18:17.101193426 +0000 UTC m=+61.396386463" observedRunningTime="2026-06-03 16:18:17.562315747 +0000 UTC m=+61.857508788" watchObservedRunningTime="2026-06-03 16:18:17.562931495 +0000 UTC m=+61.858124540" Jun 03 16:18:18.511179 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:18.511147 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-czpwp" Jun 03 16:18:21.969020 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:21.968985 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:18:21.971725 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:21.971700 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 16:18:21.982334 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:21.982306 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ac3b8bc-dd32-4bf5-97eb-84edbc66f369-metrics-certs\") pod \"network-metrics-daemon-2xsw7\" (UID: \"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369\") " pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:18:22.069783 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.069736 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:18:22.072486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.072461 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 16:18:22.082646 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.082615 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 16:18:22.093299 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.093269 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7dhz\" (UniqueName: \"kubernetes.io/projected/9a72995b-271f-4ca9-b435-57c47fa87d13-kube-api-access-z7dhz\") pod \"network-check-target-hcjql\" (UID: \"9a72995b-271f-4ca9-b435-57c47fa87d13\") " pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:18:22.227628 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.227549 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cchx6\"" Jun 03 16:18:22.232519 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.232490 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f9ccm\"" Jun 03 16:18:22.235248 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.235222 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xsw7" Jun 03 16:18:22.241017 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.240995 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:18:22.410795 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.410764 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hcjql"] Jun 03 16:18:22.414700 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:22.414665 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a72995b_271f_4ca9_b435_57c47fa87d13.slice/crio-fe6add20aaaf3f458880ebf9819493a891591ab32d038ffcd728545b845e7c55 WatchSource:0}: Error finding container fe6add20aaaf3f458880ebf9819493a891591ab32d038ffcd728545b845e7c55: Status 404 returned error can't find the container with id fe6add20aaaf3f458880ebf9819493a891591ab32d038ffcd728545b845e7c55 Jun 03 16:18:22.424078 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.424054 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2xsw7"] Jun 03 16:18:22.427008 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:22.426983 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac3b8bc_dd32_4bf5_97eb_84edbc66f369.slice/crio-1e3cfbe2aa186283ef15bfe6fa18cacda670fb9573a0ad69c34853650f29c106 WatchSource:0}: Error finding container 1e3cfbe2aa186283ef15bfe6fa18cacda670fb9573a0ad69c34853650f29c106: Status 404 returned error can't find the container with id 1e3cfbe2aa186283ef15bfe6fa18cacda670fb9573a0ad69c34853650f29c106 Jun 03 16:18:22.555895 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.555800 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xsw7" event={"ID":"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369","Type":"ContainerStarted","Data":"1e3cfbe2aa186283ef15bfe6fa18cacda670fb9573a0ad69c34853650f29c106"} Jun 03 16:18:22.557081 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:22.557045 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hcjql" event={"ID":"9a72995b-271f-4ca9-b435-57c47fa87d13","Type":"ContainerStarted","Data":"fe6add20aaaf3f458880ebf9819493a891591ab32d038ffcd728545b845e7c55"} Jun 03 16:18:26.512780 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.511483 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-444lz"] Jun 03 16:18:26.514438 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.514415 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.519280 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.519150 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 16:18:26.519280 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.519256 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 03 16:18:26.519450 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.519151 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 16:18:26.519554 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.519539 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wrrsc\"" Jun 03 16:18:26.519554 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.519547 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 03 16:18:26.519895 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.519741 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 16:18:26.529003 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.527852 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg"] Jun 03 16:18:26.530129 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.530108 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.530226 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.530112 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-444lz"] Jun 03 16:18:26.532928 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.532908 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xdwcn\"" Jun 03 16:18:26.533231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.533215 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 03 16:18:26.533400 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.533387 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 03 16:18:26.533586 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.533563 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 03 16:18:26.545260 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.542143 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg"] Jun 03 16:18:26.551013 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.550983 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ltlkt"] Jun 03 16:18:26.554749 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.554726 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.557486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.557466 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 16:18:26.557825 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.557523 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 16:18:26.557939 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.557575 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 16:18:26.559027 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.558932 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8kl4l\"" Jun 03 16:18:26.603539 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603502 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5plnk\" (UniqueName: \"kubernetes.io/projected/24a22ac6-b3b9-4472-ac95-f13671a712c1-kube-api-access-5plnk\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.603539 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-metrics-client-ca\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.603787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603565 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.603787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603590 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-root\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.603787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.603787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603650 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.603787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603675 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njx9s\" (UniqueName: \"kubernetes.io/projected/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-api-access-njx9s\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.603787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24a22ac6-b3b9-4472-ac95-f13671a712c1-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.603787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603790 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24a22ac6-b3b9-4472-ac95-f13671a712c1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603850 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-sys\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-textfile\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603919 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.603972 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-wtmp\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.604002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzb8w\" (UniqueName: \"kubernetes.io/projected/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-kube-api-access-zzb8w\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.604029 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-tls\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.604088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.604178 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.604119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/24a22ac6-b3b9-4472-ac95-f13671a712c1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.705504 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705469 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-textfile\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.705504 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705511 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705531 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-wtmp\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705556 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zzb8w\" (UniqueName: \"kubernetes.io/projected/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-kube-api-access-zzb8w\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705578 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-tls\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705596 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705616 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/24a22ac6-b3b9-4472-ac95-f13671a712c1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705640 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5plnk\" (UniqueName: \"kubernetes.io/projected/24a22ac6-b3b9-4472-ac95-f13671a712c1-kube-api-access-5plnk\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705657 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-metrics-client-ca\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705676 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705704 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-root\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705734 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705753 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.705763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705768 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-njx9s\" (UniqueName: \"kubernetes.io/projected/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-api-access-njx9s\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.706337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705793 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24a22ac6-b3b9-4472-ac95-f13671a712c1-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.706337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705814 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.706337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705833 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24a22ac6-b3b9-4472-ac95-f13671a712c1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.706337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705851 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.706337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705866 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-textfile\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.706337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705881 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-sys\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.706337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.705940 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.706337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.706045 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-wtmp\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.706706 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.706487 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-root\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.706706 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:18:26.706575 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 03 16:18:26.706706 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:18:26.706643 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-tls podName:3ef15a1d-fef4-47aa-9035-72b5d1a991f0 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:27.20662389 +0000 UTC m=+71.501816925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-tls") pod "node-exporter-ltlkt" (UID: "3ef15a1d-fef4-47aa-9035-72b5d1a991f0") : secret "node-exporter-tls" not found Jun 03 16:18:26.706706 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.706644 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.706908 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.706769 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-metrics-client-ca\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.706908 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:18:26.706861 2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 03 16:18:26.707075 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.707046 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-sys\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.707340 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.707275 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.707426 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.707404 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.708109 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.708053 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24a22ac6-b3b9-4472-ac95-f13671a712c1-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.709806 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:18:26.709785 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-tls podName:2ba6e461-7b60-47f7-befb-5fa6e2bb517b nodeName:}" failed. No retries permitted until 2026-06-03 16:18:27.206898697 +0000 UTC m=+71.502091722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-tls") pod "kube-state-metrics-8469c486bb-fxhbg" (UID: "2ba6e461-7b60-47f7-befb-5fa6e2bb517b") : secret "kube-state-metrics-tls" not found Jun 03 16:18:26.709974 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.709924 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.710236 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.710215 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.712461 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.712437 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/24a22ac6-b3b9-4472-ac95-f13671a712c1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.721201 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.721179 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5plnk\" (UniqueName: \"kubernetes.io/projected/24a22ac6-b3b9-4472-ac95-f13671a712c1-kube-api-access-5plnk\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.726725 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.726695 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzb8w\" (UniqueName: \"kubernetes.io/projected/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-kube-api-access-zzb8w\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:26.729541 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.729518 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24a22ac6-b3b9-4472-ac95-f13671a712c1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-444lz\" (UID: \"24a22ac6-b3b9-4472-ac95-f13671a712c1\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:26.729683 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.729663 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njx9s\" (UniqueName: \"kubernetes.io/projected/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-api-access-njx9s\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:26.830718 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:26.830626 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" Jun 03 16:18:27.209526 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:27.209487 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-tls\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:27.209707 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:27.209562 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:27.212192 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:27.212161 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ef15a1d-fef4-47aa-9035-72b5d1a991f0-node-exporter-tls\") pod \"node-exporter-ltlkt\" (UID: \"3ef15a1d-fef4-47aa-9035-72b5d1a991f0\") " pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:27.212324 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:27.212306 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ba6e461-7b60-47f7-befb-5fa6e2bb517b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-fxhbg\" (UID: \"2ba6e461-7b60-47f7-befb-5fa6e2bb517b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:27.447797 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:27.447754 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" Jun 03 16:18:27.469899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:27.469809 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ltlkt" Jun 03 16:18:30.947563 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.947515 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl"] Jun 03 16:18:30.951201 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.951180 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:30.953793 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.953766 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 03 16:18:30.953793 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.953783 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 03 16:18:30.954655 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.954635 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dqcj6r786vgl\"" Jun 03 16:18:30.955056 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.955015 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 03 16:18:30.955056 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.955044 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 03 16:18:30.955290 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.955276 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rn58j\"" Jun 03 16:18:30.961243 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:30.961217 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl"] Jun 03 16:18:31.037411 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.037369 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7ac826a1-68da-4dfd-aa53-297d179cc1f4-metrics-server-audit-profiles\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.037411 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.037422 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrtpq\" (UniqueName: \"kubernetes.io/projected/7ac826a1-68da-4dfd-aa53-297d179cc1f4-kube-api-access-rrtpq\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.037624 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.037517 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ac826a1-68da-4dfd-aa53-297d179cc1f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.037624 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.037564 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-secret-metrics-server-client-certs\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.037624 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.037588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-client-ca-bundle\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.037624 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.037617 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-secret-metrics-server-tls\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.037799 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.037650 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7ac826a1-68da-4dfd-aa53-297d179cc1f4-audit-log\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139048 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139006 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-secret-metrics-server-client-certs\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139056 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-client-ca-bundle\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139087 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-secret-metrics-server-tls\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139132 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7ac826a1-68da-4dfd-aa53-297d179cc1f4-audit-log\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139156 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7ac826a1-68da-4dfd-aa53-297d179cc1f4-metrics-server-audit-profiles\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139176 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rrtpq\" (UniqueName: \"kubernetes.io/projected/7ac826a1-68da-4dfd-aa53-297d179cc1f4-kube-api-access-rrtpq\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139231 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139213 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ac826a1-68da-4dfd-aa53-297d179cc1f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139739 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139711 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7ac826a1-68da-4dfd-aa53-297d179cc1f4-audit-log\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.139923 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.139886 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ac826a1-68da-4dfd-aa53-297d179cc1f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.140527 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.140478 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7ac826a1-68da-4dfd-aa53-297d179cc1f4-metrics-server-audit-profiles\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.142502 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.142477 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-client-ca-bundle\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.142830 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.142788 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-secret-metrics-server-client-certs\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.142919 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.142854 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7ac826a1-68da-4dfd-aa53-297d179cc1f4-secret-metrics-server-tls\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.148635 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.148603 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrtpq\" (UniqueName: \"kubernetes.io/projected/7ac826a1-68da-4dfd-aa53-297d179cc1f4-kube-api-access-rrtpq\") pod \"metrics-server-c8d9f5b4d-wt5zl\" (UID: \"7ac826a1-68da-4dfd-aa53-297d179cc1f4\") " pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.263056 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.262935 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:31.444381 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.439832 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-444lz"] Jun 03 16:18:31.467126 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:31.467085 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a22ac6_b3b9_4472_ac95_f13671a712c1.slice/crio-deed6e62cd3ce0c98d1965d22c084e56eab95c65211c8b45f50455a7c1503273 WatchSource:0}: Error finding container deed6e62cd3ce0c98d1965d22c084e56eab95c65211c8b45f50455a7c1503273: Status 404 returned error can't find the container with id deed6e62cd3ce0c98d1965d22c084e56eab95c65211c8b45f50455a7c1503273 Jun 03 16:18:31.478080 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.477585 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg"] Jun 03 16:18:31.483699 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:31.483208 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba6e461_7b60_47f7_befb_5fa6e2bb517b.slice/crio-36f4dbfeb551ec091c5410b2c3216a94d0334bb7db65818551fca9f72786fcd2 WatchSource:0}: Error finding container 36f4dbfeb551ec091c5410b2c3216a94d0334bb7db65818551fca9f72786fcd2: Status 404 returned error can't find the container with id 36f4dbfeb551ec091c5410b2c3216a94d0334bb7db65818551fca9f72786fcd2 Jun 03 16:18:31.511009 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.510964 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl"] Jun 03 16:18:31.516379 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:31.516345 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac826a1_68da_4dfd_aa53_297d179cc1f4.slice/crio-3decd57f91efcecaf5a87537a345b221743704712564893c50760eb6e1ce80b5 WatchSource:0}: Error finding container 3decd57f91efcecaf5a87537a345b221743704712564893c50760eb6e1ce80b5: Status 404 returned error can't find the container with id 3decd57f91efcecaf5a87537a345b221743704712564893c50760eb6e1ce80b5 Jun 03 16:18:31.584807 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.584513 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hcjql" event={"ID":"9a72995b-271f-4ca9-b435-57c47fa87d13","Type":"ContainerStarted","Data":"62087093c1f7f3cfdb417a4e92e5ff37e248e87c4b512ebfda721d4a22dfe6e6"} Jun 03 16:18:31.584807 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.584615 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:18:31.586457 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.586432 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" event={"ID":"24a22ac6-b3b9-4472-ac95-f13671a712c1","Type":"ContainerStarted","Data":"d658bd8e97af451152d113c2764e65b3e9b6f3eca33f46f3f16c1884e6864e2d"} Jun 03 16:18:31.586626 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.586613 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" event={"ID":"24a22ac6-b3b9-4472-ac95-f13671a712c1","Type":"ContainerStarted","Data":"deed6e62cd3ce0c98d1965d22c084e56eab95c65211c8b45f50455a7c1503273"} Jun 03 16:18:31.588976 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.588704 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-shkcj" event={"ID":"f534aa01-28bb-4e35-83e5-9494f96eb8de","Type":"ContainerStarted","Data":"e3d5384bbeabf874ab85ef1b11b2ce418639da77bfb93a346b0da91bc8f390a9"} Jun 03 16:18:31.589221 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.589088 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-shkcj" Jun 03 16:18:31.590897 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.590763 2563 patch_prober.go:28] interesting pod/downloads-c65b8dc4b-shkcj container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.8:8080/\": dial tcp 10.134.0.8:8080: connect: connection refused" start-of-body= Jun 03 16:18:31.590897 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.590807 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-c65b8dc4b-shkcj" podUID="f534aa01-28bb-4e35-83e5-9494f96eb8de" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.8:8080/\": dial tcp 10.134.0.8:8080: connect: connection refused" Jun 03 16:18:31.591155 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.590993 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xsw7" event={"ID":"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369","Type":"ContainerStarted","Data":"6508df0f5714ff64cf7465c9ae159dc9593e71518470c6172c45cae09df8168b"} Jun 03 16:18:31.592526 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.592498 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" event={"ID":"7ac826a1-68da-4dfd-aa53-297d179cc1f4","Type":"ContainerStarted","Data":"3decd57f91efcecaf5a87537a345b221743704712564893c50760eb6e1ce80b5"} Jun 03 16:18:31.593644 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.593600 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ltlkt" event={"ID":"3ef15a1d-fef4-47aa-9035-72b5d1a991f0","Type":"ContainerStarted","Data":"4bf4dd47b871bd93a69bcf3d1d05e738e25507b94dba0bcbf48bddb2c7ff9614"} Jun 03 16:18:31.594700 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.594680 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" event={"ID":"2ba6e461-7b60-47f7-befb-5fa6e2bb517b","Type":"ContainerStarted","Data":"36f4dbfeb551ec091c5410b2c3216a94d0334bb7db65818551fca9f72786fcd2"} Jun 03 16:18:31.607109 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.606757 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hcjql" podStartSLOduration=66.723098911 podStartE2EDuration="1m15.606739543s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:18:22.416710478 +0000 UTC m=+66.711903508" lastFinishedPulling="2026-06-03 16:18:31.300351107 +0000 UTC m=+75.595544140" observedRunningTime="2026-06-03 16:18:31.604732575 +0000 UTC m=+75.899925616" watchObservedRunningTime="2026-06-03 16:18:31.606739543 +0000 UTC m=+75.901932584" Jun 03 16:18:31.626240 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:31.626123 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2xsw7" podStartSLOduration=66.802158909 podStartE2EDuration="1m15.626089689s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:18:22.429553784 +0000 UTC m=+66.724746806" lastFinishedPulling="2026-06-03 16:18:31.253484552 +0000 UTC m=+75.548677586" observedRunningTime="2026-06-03 16:18:31.625868987 +0000 UTC m=+75.921062030" watchObservedRunningTime="2026-06-03 16:18:31.626089689 +0000 UTC m=+75.921282732" Jun 03 16:18:32.611144 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.610852 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xsw7" event={"ID":"0ac3b8bc-dd32-4bf5-97eb-84edbc66f369","Type":"ContainerStarted","Data":"29e2783370d41b0be6635262e9dcc3fc065a81864d82298850f8a8cfcf6768f9"} Jun 03 16:18:32.615721 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.615549 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" event={"ID":"24a22ac6-b3b9-4472-ac95-f13671a712c1","Type":"ContainerStarted","Data":"65e78bd4c846cc2a9bfcda09a0d7990ad17624724559ccad121901399834e5f2"} Jun 03 16:18:32.628923 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.628892 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-shkcj" Jun 03 16:18:32.670669 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.670600 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-shkcj" podStartSLOduration=2.11681084 podStartE2EDuration="19.670583744s" podCreationTimestamp="2026-06-03 16:18:13 +0000 UTC" firstStartedPulling="2026-06-03 16:18:13.767710933 +0000 UTC m=+58.062903967" lastFinishedPulling="2026-06-03 16:18:31.321483839 +0000 UTC m=+75.616676871" observedRunningTime="2026-06-03 16:18:31.650437579 +0000 UTC m=+75.945630623" watchObservedRunningTime="2026-06-03 16:18:32.670583744 +0000 UTC m=+76.965776787" Jun 03 16:18:32.782389 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.782020 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:18:32.809472 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.809426 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:18:32.809658 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.809640 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.815275 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.815241 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 03 16:18:32.815420 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.815366 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 03 16:18:32.815493 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.815439 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 03 16:18:32.815493 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.815480 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 03 16:18:32.817113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.815893 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5mw9l\"" Jun 03 16:18:32.817113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.815930 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 03 16:18:32.817113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.816022 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 03 16:18:32.817113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.816180 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 03 16:18:32.817113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.816328 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 03 16:18:32.817113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.816267 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 03 16:18:32.817113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.816503 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6fjq58cdv85h2\"" Jun 03 16:18:32.817615 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.817590 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 03 16:18:32.818717 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.818537 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 03 16:18:32.824092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.824047 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 03 16:18:32.869579 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869579 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-config\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869584 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869613 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869632 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869660 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869814 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869846 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869899 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.869922 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869924 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.870414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.869974 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.870414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.870018 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.870414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.870037 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.870414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.870059 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rrt2\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-kube-api-access-7rrt2\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.870414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.870079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.870414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.870123 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971449 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971365 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971449 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971413 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971746 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971541 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971746 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971583 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971746 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971622 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971746 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971649 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971746 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971689 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971746 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971722 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.971746 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971746 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971768 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7rrt2\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-kube-api-access-7rrt2\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971799 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971829 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971855 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971889 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-config\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971926 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.971981 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.972010 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.972053 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.972852 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.972831 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.973100 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.973076 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.973371 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.973345 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.974117 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.974090 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.978272 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.978130 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.978272 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.978231 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.978466 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.978397 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.981256 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.981228 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.982265 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.982217 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.983654 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.983139 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.983654 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.983615 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.984389 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.984347 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.984671 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.984622 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.985386 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.985347 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.986050 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.985718 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.986187 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.986160 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.987014 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.986994 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-config\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:32.991580 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:32.991560 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rrt2\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-kube-api-access-7rrt2\") pod \"prometheus-k8s-0\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:33.124733 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:33.124624 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:35.017488 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.017364 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:18:35.022687 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:18:35.022624 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e900d98_9091_45b1_b46a_4e5f3d00d818.slice/crio-287549c1265a74bb97d889d559d4bbdc307020c3b0794808fa093a0855e80d0c WatchSource:0}: Error finding container 287549c1265a74bb97d889d559d4bbdc307020c3b0794808fa093a0855e80d0c: Status 404 returned error can't find the container with id 287549c1265a74bb97d889d559d4bbdc307020c3b0794808fa093a0855e80d0c Jun 03 16:18:35.540526 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.540442 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d4f7b9487-mb7cs" Jun 03 16:18:35.629814 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.629747 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" event={"ID":"24a22ac6-b3b9-4472-ac95-f13671a712c1","Type":"ContainerStarted","Data":"14a92a80c30e3fb6ad4d769cf0b6ef793a5396113791840aa8cb19ae16db8ec6"} Jun 03 16:18:35.634989 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.633439 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" event={"ID":"7ac826a1-68da-4dfd-aa53-297d179cc1f4","Type":"ContainerStarted","Data":"a6ed32e50747a17ecbab24f0bc542a52144c8f3e26aee51f6cc4b195d7198eb6"} Jun 03 16:18:35.637066 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.636406 2563 generic.go:358] "Generic (PLEG): container finished" podID="3ef15a1d-fef4-47aa-9035-72b5d1a991f0" containerID="c698e7fc9a562f9df470cae7d85f4cc82832fe6192a57a287b7448c03a104c86" exitCode=0 Jun 03 16:18:35.637066 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.636471 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ltlkt" event={"ID":"3ef15a1d-fef4-47aa-9035-72b5d1a991f0","Type":"ContainerDied","Data":"c698e7fc9a562f9df470cae7d85f4cc82832fe6192a57a287b7448c03a104c86"} Jun 03 16:18:35.640343 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.640309 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" event={"ID":"2ba6e461-7b60-47f7-befb-5fa6e2bb517b","Type":"ContainerStarted","Data":"2b9a554479cbae0dc82279f82a672e06ba8ab7a28ccc4af3d52a8a89d1b31b4a"} Jun 03 16:18:35.640343 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.640344 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" event={"ID":"2ba6e461-7b60-47f7-befb-5fa6e2bb517b","Type":"ContainerStarted","Data":"4027bc68bd142f95fcfa33c1cf0cd7957268bbe7482c46dd95f4ae5cc25ceb46"} Jun 03 16:18:35.640650 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.640357 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" event={"ID":"2ba6e461-7b60-47f7-befb-5fa6e2bb517b","Type":"ContainerStarted","Data":"f75b115e02c98d79a211c7451731851b99c16592500e6347b3af3823c498d308"} Jun 03 16:18:35.642355 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.642334 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerStarted","Data":"287549c1265a74bb97d889d559d4bbdc307020c3b0794808fa093a0855e80d0c"} Jun 03 16:18:35.656422 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.656374 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-444lz" podStartSLOduration=6.431153484 podStartE2EDuration="9.65636072s" podCreationTimestamp="2026-06-03 16:18:26 +0000 UTC" firstStartedPulling="2026-06-03 16:18:31.605898091 +0000 UTC m=+75.901091114" lastFinishedPulling="2026-06-03 16:18:34.831105316 +0000 UTC m=+79.126298350" observedRunningTime="2026-06-03 16:18:35.655710055 +0000 UTC m=+79.950903097" watchObservedRunningTime="2026-06-03 16:18:35.65636072 +0000 UTC m=+79.951553795" Jun 03 16:18:35.700305 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.699770 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" podStartSLOduration=2.393725059 podStartE2EDuration="5.699749067s" podCreationTimestamp="2026-06-03 16:18:30 +0000 UTC" firstStartedPulling="2026-06-03 16:18:31.520297478 +0000 UTC m=+75.815490505" lastFinishedPulling="2026-06-03 16:18:34.82632148 +0000 UTC m=+79.121514513" observedRunningTime="2026-06-03 16:18:35.698158783 +0000 UTC m=+79.993351825" watchObservedRunningTime="2026-06-03 16:18:35.699749067 +0000 UTC m=+79.994942109" Jun 03 16:18:35.718143 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:35.718067 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-fxhbg" podStartSLOduration=6.377192964 podStartE2EDuration="9.718053161s" podCreationTimestamp="2026-06-03 16:18:26 +0000 UTC" firstStartedPulling="2026-06-03 16:18:31.485459897 +0000 UTC m=+75.780652917" lastFinishedPulling="2026-06-03 16:18:34.826320092 +0000 UTC m=+79.121513114" observedRunningTime="2026-06-03 16:18:35.71674067 +0000 UTC m=+80.011933706" watchObservedRunningTime="2026-06-03 16:18:35.718053161 +0000 UTC m=+80.013246202" Jun 03 16:18:36.648214 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:36.648161 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ltlkt" event={"ID":"3ef15a1d-fef4-47aa-9035-72b5d1a991f0","Type":"ContainerStarted","Data":"85329fc902213a1548ae111fbb0e9b925aeccf2bbeb66cf4b3ce0dc8b31f2224"} Jun 03 16:18:36.648214 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:36.648216 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ltlkt" event={"ID":"3ef15a1d-fef4-47aa-9035-72b5d1a991f0","Type":"ContainerStarted","Data":"f879307aa4a23c0e6eaf6d8e9f59d88238183a13ee519f5bdfa04ff81c3a9c60"} Jun 03 16:18:37.653092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:37.653054 2563 generic.go:358] "Generic (PLEG): container finished" podID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" exitCode=0 Jun 03 16:18:37.653552 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:37.653139 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerDied","Data":"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff"} Jun 03 16:18:37.701845 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:37.701780 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ltlkt" podStartSLOduration=8.167601507 podStartE2EDuration="11.70175913s" podCreationTimestamp="2026-06-03 16:18:26 +0000 UTC" firstStartedPulling="2026-06-03 16:18:31.26566934 +0000 UTC m=+75.560862364" lastFinishedPulling="2026-06-03 16:18:34.79982695 +0000 UTC m=+79.095019987" observedRunningTime="2026-06-03 16:18:36.671604884 +0000 UTC m=+80.966797929" watchObservedRunningTime="2026-06-03 16:18:37.70175913 +0000 UTC m=+81.996952173" Jun 03 16:18:41.669444 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:41.669395 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerStarted","Data":"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183"} Jun 03 16:18:41.669444 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:41.669448 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerStarted","Data":"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19"} Jun 03 16:18:43.676985 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:43.676932 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerStarted","Data":"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4"} Jun 03 16:18:43.677305 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:43.676993 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerStarted","Data":"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d"} Jun 03 16:18:44.683242 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:44.683208 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerStarted","Data":"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f"} Jun 03 16:18:44.683242 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:44.683244 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerStarted","Data":"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91"} Jun 03 16:18:44.718804 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:44.718733 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.332688093 podStartE2EDuration="12.718717703s" podCreationTimestamp="2026-06-03 16:18:32 +0000 UTC" firstStartedPulling="2026-06-03 16:18:35.024509363 +0000 UTC m=+79.319702397" lastFinishedPulling="2026-06-03 16:18:43.410538985 +0000 UTC m=+87.705732007" observedRunningTime="2026-06-03 16:18:44.716314217 +0000 UTC m=+89.011507268" watchObservedRunningTime="2026-06-03 16:18:44.718717703 +0000 UTC m=+89.013910743" Jun 03 16:18:48.124889 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:48.124844 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:18:51.264348 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:51.264314 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:18:51.264901 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:18:51.264359 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:19:02.618965 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:02.618918 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hcjql" Jun 03 16:19:11.269802 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:11.269767 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:19:11.274006 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:11.273978 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c8d9f5b4d-wt5zl" Jun 03 16:19:33.125153 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:33.125109 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:33.145500 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:33.145472 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:33.838330 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:33.838304 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:51.197295 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.197263 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:19:51.197777 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.197730 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy" containerID="cri-o://fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" gracePeriod=600 Jun 03 16:19:51.197859 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.197762 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="thanos-sidecar" containerID="cri-o://b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" gracePeriod=600 Jun 03 16:19:51.197859 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.197751 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="config-reloader" containerID="cri-o://62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" gracePeriod=600 Jun 03 16:19:51.197981 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.197716 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="prometheus" containerID="cri-o://8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" gracePeriod=600 Jun 03 16:19:51.197981 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.197865 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" gracePeriod=600 Jun 03 16:19:51.197981 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.197846 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy-web" containerID="cri-o://ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" gracePeriod=600 Jun 03 16:19:51.431785 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.431762 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:51.531392 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531302 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-config-out\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531392 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531346 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531392 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531376 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-metrics-client-ca\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531670 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531396 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-kube-rbac-proxy\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531670 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531504 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-thanos-prometheus-http-client-file\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531670 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531549 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-web-config\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531670 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531581 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531670 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531609 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-kubelet-serving-ca-bundle\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531670 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531642 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rrt2\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-kube-api-access-7rrt2\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531677 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-trusted-ca-bundle\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531709 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-rulefiles-0\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531733 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-tls-assets\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531765 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-db\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531807 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:51.531983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531818 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-config\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531890 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-metrics-client-certs\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.531983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531919 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-serving-certs-ca-bundle\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.532474 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.531989 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-tls\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.532474 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.532031 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-grpc-tls\") pod \"0e900d98-9091-45b1-b46a-4e5f3d00d818\" (UID: \"0e900d98-9091-45b1-b46a-4e5f3d00d818\") " Jun 03 16:19:51.532474 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.532444 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-metrics-client-ca\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.534512 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.532808 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:51.534512 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.532911 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:51.534512 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.533070 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:51.534512 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.534493 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:19:51.534806 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.534619 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:51.534806 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.534720 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-config" (OuterVolumeSpecName: "config") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.534806 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.534774 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.534987 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.534815 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.535415 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.535373 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.536133 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.535919 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-config-out" (OuterVolumeSpecName: "config-out") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:19:51.536609 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.536579 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-kube-api-access-7rrt2" (OuterVolumeSpecName: "kube-api-access-7rrt2") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "kube-api-access-7rrt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:19:51.536853 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.536829 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.536932 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.536867 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.536932 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.536878 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.537209 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.537177 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:19:51.537469 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.537446 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.545622 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.545599 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-web-config" (OuterVolumeSpecName: "web-config") pod "0e900d98-9091-45b1-b46a-4e5f3d00d818" (UID: "0e900d98-9091-45b1-b46a-4e5f3d00d818"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:51.633247 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633193 2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-metrics-client-certs\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633247 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633240 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633247 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633254 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633247 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633265 2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-grpc-tls\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633274 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-config-out\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633283 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633293 2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-kube-rbac-proxy\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633302 2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633311 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-web-config\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633319 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633327 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633335 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rrt2\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-kube-api-access-7rrt2\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633356 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633364 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633373 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e900d98-9091-45b1-b46a-4e5f3d00d818-tls-assets\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633381 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e900d98-9091-45b1-b46a-4e5f3d00d818-prometheus-k8s-db\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.633510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.633389 2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e900d98-9091-45b1-b46a-4e5f3d00d818-config\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:19:51.873573 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873492 2563 generic.go:358] "Generic (PLEG): container finished" podID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" exitCode=0 Jun 03 16:19:51.873573 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873515 2563 generic.go:358] "Generic (PLEG): container finished" podID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" exitCode=0 Jun 03 16:19:51.873573 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873521 2563 generic.go:358] "Generic (PLEG): container finished" podID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" exitCode=0 Jun 03 16:19:51.873573 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873528 2563 generic.go:358] "Generic (PLEG): container finished" podID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" exitCode=0 Jun 03 16:19:51.873573 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873533 2563 generic.go:358] "Generic (PLEG): container finished" podID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" exitCode=0 Jun 03 16:19:51.873573 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873537 2563 generic.go:358] "Generic (PLEG): container finished" podID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" exitCode=0 Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873572 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerDied","Data":"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f"} Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873612 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerDied","Data":"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91"} Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873623 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerDied","Data":"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4"} Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873633 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerDied","Data":"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d"} Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873622 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873642 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerDied","Data":"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183"} Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873652 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerDied","Data":"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19"} Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873662 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e900d98-9091-45b1-b46a-4e5f3d00d818","Type":"ContainerDied","Data":"287549c1265a74bb97d889d559d4bbdc307020c3b0794808fa093a0855e80d0c"} Jun 03 16:19:51.873904 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.873634 2563 scope.go:117] "RemoveContainer" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" Jun 03 16:19:51.881111 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.881096 2563 scope.go:117] "RemoveContainer" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" Jun 03 16:19:51.888635 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.888617 2563 scope.go:117] "RemoveContainer" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" Jun 03 16:19:51.897089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.897064 2563 scope.go:117] "RemoveContainer" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" Jun 03 16:19:51.898063 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.898042 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:19:51.902165 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.902142 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:19:51.905238 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.905211 2563 scope.go:117] "RemoveContainer" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" Jun 03 16:19:51.911888 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.911870 2563 scope.go:117] "RemoveContainer" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" Jun 03 16:19:51.919050 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.919032 2563 scope.go:117] "RemoveContainer" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" Jun 03 16:19:51.926182 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926156 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:19:51.926434 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926422 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="prometheus" Jun 03 16:19:51.926486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926435 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="prometheus" Jun 03 16:19:51.926486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926450 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="init-config-reloader" Jun 03 16:19:51.926486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926456 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="init-config-reloader" Jun 03 16:19:51.926486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926462 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy-web" Jun 03 16:19:51.926486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926468 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy-web" Jun 03 16:19:51.926486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926478 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy-thanos" Jun 03 16:19:51.926486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926483 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy-thanos" Jun 03 16:19:51.926486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926489 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="config-reloader" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926495 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="config-reloader" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926506 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="thanos-sidecar" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926511 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="thanos-sidecar" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926517 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926548 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926605 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="config-reloader" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926614 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926621 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="prometheus" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926627 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="thanos-sidecar" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926634 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy-web" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926639 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" containerName="kube-rbac-proxy-thanos" Jun 03 16:19:51.926753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.926706 2563 scope.go:117] "RemoveContainer" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" Jun 03 16:19:51.927259 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:19:51.927126 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": container with ID starting with ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f not found: ID does not exist" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" Jun 03 16:19:51.927259 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.927162 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f"} err="failed to get container status \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": rpc error: code = NotFound desc = could not find container \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": container with ID starting with ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f not found: ID does not exist" Jun 03 16:19:51.927259 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.927210 2563 scope.go:117] "RemoveContainer" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" Jun 03 16:19:51.927510 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:19:51.927487 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": container with ID starting with fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91 not found: ID does not exist" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" Jun 03 16:19:51.927553 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.927516 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91"} err="failed to get container status \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": rpc error: code = NotFound desc = could not find container \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": container with ID starting with fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91 not found: ID does not exist" Jun 03 16:19:51.927553 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.927533 2563 scope.go:117] "RemoveContainer" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" Jun 03 16:19:51.927781 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:19:51.927757 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": container with ID starting with ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4 not found: ID does not exist" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" Jun 03 16:19:51.927816 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.927792 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4"} err="failed to get container status \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": rpc error: code = NotFound desc = could not find container \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": container with ID starting with ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4 not found: ID does not exist" Jun 03 16:19:51.927860 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.927816 2563 scope.go:117] "RemoveContainer" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" Jun 03 16:19:51.928066 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:19:51.928051 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": container with ID starting with b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d not found: ID does not exist" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" Jun 03 16:19:51.928103 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.928072 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d"} err="failed to get container status \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": rpc error: code = NotFound desc = could not find container \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": container with ID starting with b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d not found: ID does not exist" Jun 03 16:19:51.928103 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.928087 2563 scope.go:117] "RemoveContainer" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" Jun 03 16:19:51.928316 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:19:51.928301 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": container with ID starting with 62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183 not found: ID does not exist" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" Jun 03 16:19:51.928358 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.928322 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183"} err="failed to get container status \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": rpc error: code = NotFound desc = could not find container \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": container with ID starting with 62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183 not found: ID does not exist" Jun 03 16:19:51.928358 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.928336 2563 scope.go:117] "RemoveContainer" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" Jun 03 16:19:51.928566 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:19:51.928550 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": container with ID starting with 8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19 not found: ID does not exist" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" Jun 03 16:19:51.928619 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.928572 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19"} err="failed to get container status \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": rpc error: code = NotFound desc = could not find container \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": container with ID starting with 8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19 not found: ID does not exist" Jun 03 16:19:51.928619 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.928587 2563 scope.go:117] "RemoveContainer" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" Jun 03 16:19:51.928811 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:19:51.928794 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": container with ID starting with e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff not found: ID does not exist" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" Jun 03 16:19:51.928851 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.928815 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff"} err="failed to get container status \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": rpc error: code = NotFound desc = could not find container \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": container with ID starting with e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff not found: ID does not exist" Jun 03 16:19:51.928851 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.928827 2563 scope.go:117] "RemoveContainer" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" Jun 03 16:19:51.929056 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929032 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f"} err="failed to get container status \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": rpc error: code = NotFound desc = could not find container \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": container with ID starting with ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f not found: ID does not exist" Jun 03 16:19:51.929111 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929059 2563 scope.go:117] "RemoveContainer" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" Jun 03 16:19:51.929275 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929259 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91"} err="failed to get container status \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": rpc error: code = NotFound desc = could not find container \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": container with ID starting with fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91 not found: ID does not exist" Jun 03 16:19:51.929318 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929275 2563 scope.go:117] "RemoveContainer" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" Jun 03 16:19:51.929501 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929484 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4"} err="failed to get container status \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": rpc error: code = NotFound desc = could not find container \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": container with ID starting with ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4 not found: ID does not exist" Jun 03 16:19:51.929501 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929500 2563 scope.go:117] "RemoveContainer" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" Jun 03 16:19:51.929689 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929674 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d"} err="failed to get container status \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": rpc error: code = NotFound desc = could not find container \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": container with ID starting with b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d not found: ID does not exist" Jun 03 16:19:51.929736 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929689 2563 scope.go:117] "RemoveContainer" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" Jun 03 16:19:51.929863 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929848 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183"} err="failed to get container status \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": rpc error: code = NotFound desc = could not find container \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": container with ID starting with 62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183 not found: ID does not exist" Jun 03 16:19:51.929911 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.929863 2563 scope.go:117] "RemoveContainer" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" Jun 03 16:19:51.930074 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930053 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19"} err="failed to get container status \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": rpc error: code = NotFound desc = could not find container \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": container with ID starting with 8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19 not found: ID does not exist" Jun 03 16:19:51.930143 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930075 2563 scope.go:117] "RemoveContainer" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" Jun 03 16:19:51.930287 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930269 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff"} err="failed to get container status \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": rpc error: code = NotFound desc = could not find container \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": container with ID starting with e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff not found: ID does not exist" Jun 03 16:19:51.930345 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930288 2563 scope.go:117] "RemoveContainer" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" Jun 03 16:19:51.930470 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930454 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f"} err="failed to get container status \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": rpc error: code = NotFound desc = could not find container \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": container with ID starting with ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f not found: ID does not exist" Jun 03 16:19:51.930519 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930471 2563 scope.go:117] "RemoveContainer" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" Jun 03 16:19:51.930685 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930666 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91"} err="failed to get container status \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": rpc error: code = NotFound desc = could not find container \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": container with ID starting with fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91 not found: ID does not exist" Jun 03 16:19:51.930754 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930687 2563 scope.go:117] "RemoveContainer" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" Jun 03 16:19:51.930754 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930712 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:51.931024 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930908 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4"} err="failed to get container status \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": rpc error: code = NotFound desc = could not find container \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": container with ID starting with ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4 not found: ID does not exist" Jun 03 16:19:51.931024 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.930933 2563 scope.go:117] "RemoveContainer" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" Jun 03 16:19:51.931382 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.931354 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d"} err="failed to get container status \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": rpc error: code = NotFound desc = could not find container \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": container with ID starting with b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d not found: ID does not exist" Jun 03 16:19:51.931460 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.931383 2563 scope.go:117] "RemoveContainer" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" Jun 03 16:19:51.931665 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.931632 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183"} err="failed to get container status \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": rpc error: code = NotFound desc = could not find container \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": container with ID starting with 62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183 not found: ID does not exist" Jun 03 16:19:51.931665 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.931665 2563 scope.go:117] "RemoveContainer" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" Jun 03 16:19:51.932002 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.931983 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19"} err="failed to get container status \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": rpc error: code = NotFound desc = could not find container \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": container with ID starting with 8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19 not found: ID does not exist" Jun 03 16:19:51.932002 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932008 2563 scope.go:117] "RemoveContainer" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" Jun 03 16:19:51.932226 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932206 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff"} err="failed to get container status \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": rpc error: code = NotFound desc = could not find container \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": container with ID starting with e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff not found: ID does not exist" Jun 03 16:19:51.932270 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932227 2563 scope.go:117] "RemoveContainer" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" Jun 03 16:19:51.932414 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932399 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f"} err="failed to get container status \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": rpc error: code = NotFound desc = could not find container \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": container with ID starting with ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f not found: ID does not exist" Jun 03 16:19:51.932454 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932414 2563 scope.go:117] "RemoveContainer" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" Jun 03 16:19:51.932579 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932564 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91"} err="failed to get container status \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": rpc error: code = NotFound desc = could not find container \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": container with ID starting with fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91 not found: ID does not exist" Jun 03 16:19:51.932624 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932580 2563 scope.go:117] "RemoveContainer" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" Jun 03 16:19:51.932802 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932787 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4"} err="failed to get container status \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": rpc error: code = NotFound desc = could not find container \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": container with ID starting with ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4 not found: ID does not exist" Jun 03 16:19:51.932845 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.932802 2563 scope.go:117] "RemoveContainer" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" Jun 03 16:19:51.933037 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933016 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d"} err="failed to get container status \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": rpc error: code = NotFound desc = could not find container \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": container with ID starting with b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d not found: ID does not exist" Jun 03 16:19:51.933085 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933038 2563 scope.go:117] "RemoveContainer" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" Jun 03 16:19:51.933301 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933280 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183"} err="failed to get container status \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": rpc error: code = NotFound desc = could not find container \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": container with ID starting with 62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183 not found: ID does not exist" Jun 03 16:19:51.933392 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933304 2563 scope.go:117] "RemoveContainer" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" Jun 03 16:19:51.933532 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933514 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19"} err="failed to get container status \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": rpc error: code = NotFound desc = could not find container \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": container with ID starting with 8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19 not found: ID does not exist" Jun 03 16:19:51.933571 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933533 2563 scope.go:117] "RemoveContainer" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" Jun 03 16:19:51.933757 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933735 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff"} err="failed to get container status \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": rpc error: code = NotFound desc = could not find container \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": container with ID starting with e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff not found: ID does not exist" Jun 03 16:19:51.933832 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933760 2563 scope.go:117] "RemoveContainer" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" Jun 03 16:19:51.933895 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933842 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 03 16:19:51.933895 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933861 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 03 16:19:51.934043 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933896 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 03 16:19:51.934043 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933905 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6fjq58cdv85h2\"" Jun 03 16:19:51.934043 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933923 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 03 16:19:51.934043 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933863 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 03 16:19:51.934043 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.933842 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 03 16:19:51.934043 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934037 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5mw9l\"" Jun 03 16:19:51.934329 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934269 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 03 16:19:51.934329 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934291 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f"} err="failed to get container status \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": rpc error: code = NotFound desc = could not find container \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": container with ID starting with ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f not found: ID does not exist" Jun 03 16:19:51.934329 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934314 2563 scope.go:117] "RemoveContainer" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" Jun 03 16:19:51.934529 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934423 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 03 16:19:51.934529 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934424 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 03 16:19:51.934529 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934534 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 03 16:19:51.934686 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934563 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91"} err="failed to get container status \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": rpc error: code = NotFound desc = could not find container \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": container with ID starting with fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91 not found: ID does not exist" Jun 03 16:19:51.934686 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934584 2563 scope.go:117] "RemoveContainer" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" Jun 03 16:19:51.934881 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934830 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4"} err="failed to get container status \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": rpc error: code = NotFound desc = could not find container \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": container with ID starting with ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4 not found: ID does not exist" Jun 03 16:19:51.935014 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.934882 2563 scope.go:117] "RemoveContainer" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" Jun 03 16:19:51.935216 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.935190 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d"} err="failed to get container status \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": rpc error: code = NotFound desc = could not find container \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": container with ID starting with b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d not found: ID does not exist" Jun 03 16:19:51.935311 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.935220 2563 scope.go:117] "RemoveContainer" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" Jun 03 16:19:51.935492 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.935467 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183"} err="failed to get container status \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": rpc error: code = NotFound desc = could not find container \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": container with ID starting with 62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183 not found: ID does not exist" Jun 03 16:19:51.935553 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.935500 2563 scope.go:117] "RemoveContainer" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" Jun 03 16:19:51.935744 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.935713 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19"} err="failed to get container status \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": rpc error: code = NotFound desc = could not find container \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": container with ID starting with 8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19 not found: ID does not exist" Jun 03 16:19:51.935744 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.935742 2563 scope.go:117] "RemoveContainer" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" Jun 03 16:19:51.936017 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.935980 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff"} err="failed to get container status \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": rpc error: code = NotFound desc = could not find container \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": container with ID starting with e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff not found: ID does not exist" Jun 03 16:19:51.936091 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.936026 2563 scope.go:117] "RemoveContainer" containerID="ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f" Jun 03 16:19:51.936271 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.936251 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f"} err="failed to get container status \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": rpc error: code = NotFound desc = could not find container \"ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f\": container with ID starting with ab237d7b20ced2d25231d28e56388f9332c1974c346a46c0e945e2f946e6071f not found: ID does not exist" Jun 03 16:19:51.936337 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.936274 2563 scope.go:117] "RemoveContainer" containerID="fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91" Jun 03 16:19:51.936506 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.936489 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91"} err="failed to get container status \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": rpc error: code = NotFound desc = could not find container \"fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91\": container with ID starting with fc00b00de55311103fac423c062a9a76059b6b3b7ebb710f91e0b3276592ce91 not found: ID does not exist" Jun 03 16:19:51.936580 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.936507 2563 scope.go:117] "RemoveContainer" containerID="ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4" Jun 03 16:19:51.936785 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.936761 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4"} err="failed to get container status \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": rpc error: code = NotFound desc = could not find container \"ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4\": container with ID starting with ee31e953fe6c9f30f418a479e7e837c854a748d8d5fbf54762751212928056d4 not found: ID does not exist" Jun 03 16:19:51.936861 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.936787 2563 scope.go:117] "RemoveContainer" containerID="b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d" Jun 03 16:19:51.937093 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.937071 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d"} err="failed to get container status \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": rpc error: code = NotFound desc = could not find container \"b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d\": container with ID starting with b2670efa49cfd54f5bc65efc901345028b5bfd916fa152e115afa94460e1720d not found: ID does not exist" Jun 03 16:19:51.937163 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.937094 2563 scope.go:117] "RemoveContainer" containerID="62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183" Jun 03 16:19:51.937420 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.937371 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183"} err="failed to get container status \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": rpc error: code = NotFound desc = could not find container \"62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183\": container with ID starting with 62b16a43cadb93518176cb82b373b7bfa70fa5bd37ee08ac18f6363cc955b183 not found: ID does not exist" Jun 03 16:19:51.937420 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.937398 2563 scope.go:117] "RemoveContainer" containerID="8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19" Jun 03 16:19:51.937420 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.937438 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 03 16:19:51.937835 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.937810 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19"} err="failed to get container status \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": rpc error: code = NotFound desc = could not find container \"8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19\": container with ID starting with 8cd102699f4fd51d94927c135dfc95d6e5caca4b49f131db98f0112e3b934c19 not found: ID does not exist" Jun 03 16:19:51.937891 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.937837 2563 scope.go:117] "RemoveContainer" containerID="e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff" Jun 03 16:19:51.938131 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.938108 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff"} err="failed to get container status \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": rpc error: code = NotFound desc = could not find container \"e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff\": container with ID starting with e552cf1e0ab7ad9f88fd1bdad3be93a2126d62b5b606ddb334a7d1bfde9c0dff not found: ID does not exist" Jun 03 16:19:51.940926 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.940907 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 03 16:19:51.944557 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:51.944533 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:19:52.037112 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037073 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037285 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037117 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037285 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037148 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-config\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037285 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037195 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037285 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037225 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037285 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037273 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037437 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037295 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037437 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037323 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037437 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037369 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037437 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037385 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037437 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037404 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037437 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037427 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jml5z\" (UniqueName: \"kubernetes.io/projected/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-kube-api-access-jml5z\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037612 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037466 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037612 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037612 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037518 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037612 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037532 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037612 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037548 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.037749 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.037608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138511 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138479 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138658 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138520 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138658 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138542 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138730 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138661 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138730 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138701 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138826 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138731 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138826 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138756 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138826 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138785 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jml5z\" (UniqueName: \"kubernetes.io/projected/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-kube-api-access-jml5z\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.138826 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138811 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139023 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138859 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139023 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138886 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139023 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138911 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139023 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.138939 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139023 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.139015 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139227 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.139050 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139227 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.139080 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139227 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.139112 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-config\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139227 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.139175 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139227 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.139177 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139467 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.139297 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.139467 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.139360 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.142379 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.142067 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.142379 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.142148 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.142379 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.142256 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.142379 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.142279 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.142689 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.142428 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.142689 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.142514 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.142689 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.142284 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.142901 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.142751 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.143404 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.143381 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.143494 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.143385 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.144491 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.144470 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.144566 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.144510 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-config\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.144937 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.144922 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.145607 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.145588 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.150834 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.150812 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jml5z\" (UniqueName: \"kubernetes.io/projected/2abe109e-4130-41ce-b2c0-038c8cf4b1bc-kube-api-access-jml5z\") pod \"prometheus-k8s-0\" (UID: \"2abe109e-4130-41ce-b2c0-038c8cf4b1bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.242212 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.242179 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:19:52.316471 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.316441 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e900d98-9091-45b1-b46a-4e5f3d00d818" path="/var/lib/kubelet/pods/0e900d98-9091-45b1-b46a-4e5f3d00d818/volumes" Jun 03 16:19:52.378180 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.378145 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:19:52.381265 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:19:52.381236 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2abe109e_4130_41ce_b2c0_038c8cf4b1bc.slice/crio-43e11877bde556e3e5a0d530d1869f8b45c40d3080a245e2eb5f358be90e7e48 WatchSource:0}: Error finding container 43e11877bde556e3e5a0d530d1869f8b45c40d3080a245e2eb5f358be90e7e48: Status 404 returned error can't find the container with id 43e11877bde556e3e5a0d530d1869f8b45c40d3080a245e2eb5f358be90e7e48 Jun 03 16:19:52.878427 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.878388 2563 generic.go:358] "Generic (PLEG): container finished" podID="2abe109e-4130-41ce-b2c0-038c8cf4b1bc" containerID="d6748b76931e20cee8abb70637f38c3119abf00cd06a655eb5490620df072c71" exitCode=0 Jun 03 16:19:52.878591 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.878431 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2abe109e-4130-41ce-b2c0-038c8cf4b1bc","Type":"ContainerDied","Data":"d6748b76931e20cee8abb70637f38c3119abf00cd06a655eb5490620df072c71"} Jun 03 16:19:52.878591 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:52.878452 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2abe109e-4130-41ce-b2c0-038c8cf4b1bc","Type":"ContainerStarted","Data":"43e11877bde556e3e5a0d530d1869f8b45c40d3080a245e2eb5f358be90e7e48"} Jun 03 16:19:53.883584 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:53.883545 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2abe109e-4130-41ce-b2c0-038c8cf4b1bc","Type":"ContainerStarted","Data":"7b7765729c8f85748a2b1381e45eca73144b8057484aee95672fd6f44606747e"} Jun 03 16:19:53.883584 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:53.883584 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2abe109e-4130-41ce-b2c0-038c8cf4b1bc","Type":"ContainerStarted","Data":"dd4698d9ebcf5ddfbcbc1a9bd2b9564aaa092bb360fc0f2c91eca4734be1306a"} Jun 03 16:19:53.884081 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:53.883598 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2abe109e-4130-41ce-b2c0-038c8cf4b1bc","Type":"ContainerStarted","Data":"ddeaaf4201380e9afae0e4dfbbf9abb36d8a9ea5b3a84f13e1fa4cf6011ddc1e"} Jun 03 16:19:53.884081 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:53.883611 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2abe109e-4130-41ce-b2c0-038c8cf4b1bc","Type":"ContainerStarted","Data":"153d8df9e7bf8c19718ed91613acef750e002f5081563002b786ad916cc80b97"} Jun 03 16:19:53.884081 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:53.883621 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2abe109e-4130-41ce-b2c0-038c8cf4b1bc","Type":"ContainerStarted","Data":"ef1bce65813ad9b8c31f814bbee4c9d0dcd0d8c4c030f189c861b0e53ab2907f"} Jun 03 16:19:53.884081 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:53.883631 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2abe109e-4130-41ce-b2c0-038c8cf4b1bc","Type":"ContainerStarted","Data":"63377f1bf7b13071c5cdf650a1b6c3724f61eb16431cfc75b0f376b5960e0876"} Jun 03 16:19:53.911908 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:53.911856 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.911840883 podStartE2EDuration="2.911840883s" podCreationTimestamp="2026-06-03 16:19:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:19:53.911089949 +0000 UTC m=+158.206282989" watchObservedRunningTime="2026-06-03 16:19:53.911840883 +0000 UTC m=+158.207033924" Jun 03 16:19:57.243141 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:19:57.243099 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:20:52.243025 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:20:52.242980 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:20:52.259360 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:20:52.259332 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:20:53.061229 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:20:53.061199 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:21:48.065563 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.065524 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-gx7b4"] Jun 03 16:21:48.067639 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.067621 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.069843 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.069822 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 16:21:48.075723 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.075695 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gx7b4"] Jun 03 16:21:48.163965 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.163920 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-dbus\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.164181 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.163994 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-original-pull-secret\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.164181 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.164024 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-kubelet-config\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.264761 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.264722 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-dbus\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.264761 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.264766 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-original-pull-secret\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.265036 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.264785 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-kubelet-config\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.265036 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.264877 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-kubelet-config\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.265036 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.264977 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-dbus\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.267285 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.267258 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/618bb3a0-8cda-4675-a219-c4b2aefb8f0f-original-pull-secret\") pod \"global-pull-secret-syncer-gx7b4\" (UID: \"618bb3a0-8cda-4675-a219-c4b2aefb8f0f\") " pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.377834 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.377804 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gx7b4" Jun 03 16:21:48.506911 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:48.506876 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gx7b4"] Jun 03 16:21:48.510196 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:21:48.510165 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod618bb3a0_8cda_4675_a219_c4b2aefb8f0f.slice/crio-210c79a353db3f73069967b55d0c26e7d4456c4fb5b1dc4fc171224c0f803178 WatchSource:0}: Error finding container 210c79a353db3f73069967b55d0c26e7d4456c4fb5b1dc4fc171224c0f803178: Status 404 returned error can't find the container with id 210c79a353db3f73069967b55d0c26e7d4456c4fb5b1dc4fc171224c0f803178 Jun 03 16:21:49.197838 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:49.197798 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gx7b4" event={"ID":"618bb3a0-8cda-4675-a219-c4b2aefb8f0f","Type":"ContainerStarted","Data":"210c79a353db3f73069967b55d0c26e7d4456c4fb5b1dc4fc171224c0f803178"} Jun 03 16:21:53.211875 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:53.211835 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gx7b4" event={"ID":"618bb3a0-8cda-4675-a219-c4b2aefb8f0f","Type":"ContainerStarted","Data":"8acf055a9b7c3501573a22d28e4c100c5dbbd3966d6d4e482e722c80cc2d36b3"} Jun 03 16:21:53.228474 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:21:53.228415 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gx7b4" podStartSLOduration=0.99573106 podStartE2EDuration="5.228397003s" podCreationTimestamp="2026-06-03 16:21:48 +0000 UTC" firstStartedPulling="2026-06-03 16:21:48.513939495 +0000 UTC m=+272.809132516" lastFinishedPulling="2026-06-03 16:21:52.74660544 +0000 UTC m=+277.041798459" observedRunningTime="2026-06-03 16:21:53.227251184 +0000 UTC m=+277.522444225" watchObservedRunningTime="2026-06-03 16:21:53.228397003 +0000 UTC m=+277.523590073" Jun 03 16:22:16.185731 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:22:16.185702 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:22:16.186311 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:22:16.186138 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:22:16.191307 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:22:16.191282 2563 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 16:23:34.125298 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.125262 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz"] Jun 03 16:23:34.127484 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.127468 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.129992 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.129922 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 03 16:23:34.130181 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.130157 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lbsp8\"" Jun 03 16:23:34.130257 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.130185 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 03 16:23:34.130257 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.130212 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 03 16:23:34.130347 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.130340 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 03 16:23:34.143018 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.142975 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz"] Jun 03 16:23:34.289666 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.289623 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp82x\" (UniqueName: \"kubernetes.io/projected/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-kube-api-access-tp82x\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.289666 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.289670 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.289901 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.289752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.390228 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.390191 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.390425 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.390260 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tp82x\" (UniqueName: \"kubernetes.io/projected/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-kube-api-access-tp82x\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.390425 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.390278 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.393004 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.392978 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.393095 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.392978 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.399618 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.399594 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp82x\" (UniqueName: \"kubernetes.io/projected/16dc22cf-d4e4-41e8-9beb-7b2f3217a46f-kube-api-access-tp82x\") pod \"opendatahub-operator-controller-manager-6c656db94d-k6nfz\" (UID: \"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.437926 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.437892 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:34.580503 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.580463 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz"] Jun 03 16:23:34.583640 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:23:34.583611 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16dc22cf_d4e4_41e8_9beb_7b2f3217a46f.slice/crio-e4bcf1a065c6dfbd62bb3ee7ef533fa9a7a1316ad1006cdde51b91bf250ee2d7 WatchSource:0}: Error finding container e4bcf1a065c6dfbd62bb3ee7ef533fa9a7a1316ad1006cdde51b91bf250ee2d7: Status 404 returned error can't find the container with id e4bcf1a065c6dfbd62bb3ee7ef533fa9a7a1316ad1006cdde51b91bf250ee2d7 Jun 03 16:23:34.585254 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:34.585236 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:23:35.487579 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:35.487538 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" event={"ID":"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f","Type":"ContainerStarted","Data":"e4bcf1a065c6dfbd62bb3ee7ef533fa9a7a1316ad1006cdde51b91bf250ee2d7"} Jun 03 16:23:37.494739 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:37.494653 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" event={"ID":"16dc22cf-d4e4-41e8-9beb-7b2f3217a46f","Type":"ContainerStarted","Data":"fd26df788cae08881dbdfc5f811ddbd804d982ab8bacb3e31cba002d7b9ed095"} Jun 03 16:23:37.495106 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:37.494742 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:37.545312 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:37.545256 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" podStartSLOduration=0.932187612 podStartE2EDuration="3.545241053s" podCreationTimestamp="2026-06-03 16:23:34 +0000 UTC" firstStartedPulling="2026-06-03 16:23:34.585366458 +0000 UTC m=+378.880559479" lastFinishedPulling="2026-06-03 16:23:37.198419884 +0000 UTC m=+381.493612920" observedRunningTime="2026-06-03 16:23:37.543621986 +0000 UTC m=+381.838815026" watchObservedRunningTime="2026-06-03 16:23:37.545241053 +0000 UTC m=+381.840434094" Jun 03 16:23:39.975577 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.975541 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk"] Jun 03 16:23:39.978132 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.978113 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:39.987219 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.987199 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 03 16:23:39.988211 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.988185 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 03 16:23:39.988211 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.988200 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:23:39.988378 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.988257 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 03 16:23:39.988378 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.988331 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tm59q\"" Jun 03 16:23:39.993576 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.993549 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk"] Jun 03 16:23:39.997482 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:39.997462 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 03 16:23:40.033113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.033079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-cert\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.033306 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.033121 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-metrics-cert\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.033306 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.033209 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w45np\" (UniqueName: \"kubernetes.io/projected/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-kube-api-access-w45np\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.033306 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.033265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-manager-config\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.134118 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.134074 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w45np\" (UniqueName: \"kubernetes.io/projected/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-kube-api-access-w45np\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.134310 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.134130 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-manager-config\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.134310 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.134166 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-cert\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.134310 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.134181 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-metrics-cert\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.134844 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.134817 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-manager-config\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.136818 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.136786 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-metrics-cert\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.136909 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.136844 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-cert\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.142624 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.142599 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w45np\" (UniqueName: \"kubernetes.io/projected/5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b-kube-api-access-w45np\") pod \"lws-controller-manager-bd65fcf99-4vfjk\" (UID: \"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.287762 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.287673 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:40.438807 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.438773 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk"] Jun 03 16:23:40.441840 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:23:40.441797 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce1b328_c5bd_4a3c_97ed_9e6356a55b6b.slice/crio-10b9d51a186f60c6afcea5b396ba514f07457208812019068f95a836bac575f7 WatchSource:0}: Error finding container 10b9d51a186f60c6afcea5b396ba514f07457208812019068f95a836bac575f7: Status 404 returned error can't find the container with id 10b9d51a186f60c6afcea5b396ba514f07457208812019068f95a836bac575f7 Jun 03 16:23:40.504535 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:40.504499 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" event={"ID":"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b","Type":"ContainerStarted","Data":"10b9d51a186f60c6afcea5b396ba514f07457208812019068f95a836bac575f7"} Jun 03 16:23:44.518220 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:44.518174 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" event={"ID":"5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b","Type":"ContainerStarted","Data":"67844c8bd0411281a9a4ce21a2fa9fdb2da805c05a750ec5da65bbbf2921eccb"} Jun 03 16:23:44.518707 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:44.518321 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:23:44.535869 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:44.535815 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" podStartSLOduration=2.516139746 podStartE2EDuration="5.535799092s" podCreationTimestamp="2026-06-03 16:23:39 +0000 UTC" firstStartedPulling="2026-06-03 16:23:40.443589552 +0000 UTC m=+384.738782571" lastFinishedPulling="2026-06-03 16:23:43.463248894 +0000 UTC m=+387.758441917" observedRunningTime="2026-06-03 16:23:44.53476979 +0000 UTC m=+388.829962828" watchObservedRunningTime="2026-06-03 16:23:44.535799092 +0000 UTC m=+388.830992216" Jun 03 16:23:48.500289 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:48.500259 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-k6nfz" Jun 03 16:23:55.523711 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:23:55.523668 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-4vfjk" Jun 03 16:24:03.162970 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.160901 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm"] Jun 03 16:24:03.170563 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.170536 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.173045 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.173017 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 16:24:03.173184 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.173128 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 03 16:24:03.173288 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.173273 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 03 16:24:03.173343 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.173273 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 16:24:03.173799 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.173781 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sgdvz\"" Jun 03 16:24:03.181834 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.181805 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm"] Jun 03 16:24:03.235976 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.235923 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91b33d16-f254-4940-8265-039e65e0059a-tls-certs\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.236166 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.235987 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvtf7\" (UniqueName: \"kubernetes.io/projected/91b33d16-f254-4940-8265-039e65e0059a-kube-api-access-bvtf7\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.236166 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.236089 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91b33d16-f254-4940-8265-039e65e0059a-tmp\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.336877 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.336833 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91b33d16-f254-4940-8265-039e65e0059a-tmp\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.337065 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.336917 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91b33d16-f254-4940-8265-039e65e0059a-tls-certs\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.337065 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.336980 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bvtf7\" (UniqueName: \"kubernetes.io/projected/91b33d16-f254-4940-8265-039e65e0059a-kube-api-access-bvtf7\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.337065 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:24:03.337051 2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 03 16:24:03.337186 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:24:03.337146 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b33d16-f254-4940-8265-039e65e0059a-tls-certs podName:91b33d16-f254-4940-8265-039e65e0059a nodeName:}" failed. No retries permitted until 2026-06-03 16:24:03.837126504 +0000 UTC m=+408.132319535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/91b33d16-f254-4940-8265-039e65e0059a-tls-certs") pod "kube-auth-proxy-7b4fb99578-2dwtm" (UID: "91b33d16-f254-4940-8265-039e65e0059a") : secret "kube-auth-proxy-tls" not found Jun 03 16:24:03.339282 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.339260 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91b33d16-f254-4940-8265-039e65e0059a-tmp\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.345346 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.345326 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvtf7\" (UniqueName: \"kubernetes.io/projected/91b33d16-f254-4940-8265-039e65e0059a-kube-api-access-bvtf7\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.840912 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.840807 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91b33d16-f254-4940-8265-039e65e0059a-tls-certs\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:03.843375 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:03.843343 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91b33d16-f254-4940-8265-039e65e0059a-tls-certs\") pod \"kube-auth-proxy-7b4fb99578-2dwtm\" (UID: \"91b33d16-f254-4940-8265-039e65e0059a\") " pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:04.082021 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:04.081985 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" Jun 03 16:24:04.213498 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:04.213326 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm"] Jun 03 16:24:04.216054 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:24:04.216014 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b33d16_f254_4940_8265_039e65e0059a.slice/crio-1cfa22af937e451e382b09afdd36f1f6df0affa5e46b65b13a435a23d4b7cdfd WatchSource:0}: Error finding container 1cfa22af937e451e382b09afdd36f1f6df0affa5e46b65b13a435a23d4b7cdfd: Status 404 returned error can't find the container with id 1cfa22af937e451e382b09afdd36f1f6df0affa5e46b65b13a435a23d4b7cdfd Jun 03 16:24:04.574588 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:04.574493 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" event={"ID":"91b33d16-f254-4940-8265-039e65e0059a","Type":"ContainerStarted","Data":"1cfa22af937e451e382b09afdd36f1f6df0affa5e46b65b13a435a23d4b7cdfd"} Jun 03 16:24:09.591170 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:09.591129 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" event={"ID":"91b33d16-f254-4940-8265-039e65e0059a","Type":"ContainerStarted","Data":"93dcb4c3141a254e2e59742a51aed438338ca4a94804e1c71a92b6a140d039e7"} Jun 03 16:24:09.607468 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:24:09.607415 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b4fb99578-2dwtm" podStartSLOduration=1.9628456220000001 podStartE2EDuration="6.60740266s" podCreationTimestamp="2026-06-03 16:24:03 +0000 UTC" firstStartedPulling="2026-06-03 16:24:04.217838621 +0000 UTC m=+408.513031640" lastFinishedPulling="2026-06-03 16:24:08.862395655 +0000 UTC m=+413.157588678" observedRunningTime="2026-06-03 16:24:09.606470936 +0000 UTC m=+413.901663978" watchObservedRunningTime="2026-06-03 16:24:09.60740266 +0000 UTC m=+413.902595745" Jun 03 16:25:36.911162 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:36.911125 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl"] Jun 03 16:25:36.914594 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:36.914572 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:36.917541 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:36.917513 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lctwc\"" Jun 03 16:25:36.917658 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:36.917524 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 16:25:36.918329 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:36.918312 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 16:25:36.937228 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:36.937203 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl"] Jun 03 16:25:37.022551 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.022509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgxpw\" (UniqueName: \"kubernetes.io/projected/db1b88a7-2302-436d-a3dc-a3f1f135a40d-kube-api-access-kgxpw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" (UID: \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:37.022748 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.022572 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db1b88a7-2302-436d-a3dc-a3f1f135a40d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" (UID: \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:37.124113 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.124078 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgxpw\" (UniqueName: \"kubernetes.io/projected/db1b88a7-2302-436d-a3dc-a3f1f135a40d-kube-api-access-kgxpw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" (UID: \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:37.124258 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.124134 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db1b88a7-2302-436d-a3dc-a3f1f135a40d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" (UID: \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:37.124494 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.124477 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db1b88a7-2302-436d-a3dc-a3f1f135a40d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" (UID: \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:37.132780 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.132750 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgxpw\" (UniqueName: \"kubernetes.io/projected/db1b88a7-2302-436d-a3dc-a3f1f135a40d-kube-api-access-kgxpw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" (UID: \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:37.225064 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.224968 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:37.360469 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.360409 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl"] Jun 03 16:25:37.364247 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:25:37.364203 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1b88a7_2302_436d_a3dc_a3f1f135a40d.slice/crio-baed811bef2e8f0f56848ac16adddc96b5f08d9a671805e0e1f73c18e77c06f8 WatchSource:0}: Error finding container baed811bef2e8f0f56848ac16adddc96b5f08d9a671805e0e1f73c18e77c06f8: Status 404 returned error can't find the container with id baed811bef2e8f0f56848ac16adddc96b5f08d9a671805e0e1f73c18e77c06f8 Jun 03 16:25:37.843239 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:37.843202 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" event={"ID":"db1b88a7-2302-436d-a3dc-a3f1f135a40d","Type":"ContainerStarted","Data":"baed811bef2e8f0f56848ac16adddc96b5f08d9a671805e0e1f73c18e77c06f8"} Jun 03 16:25:43.864282 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:43.864244 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" event={"ID":"db1b88a7-2302-436d-a3dc-a3f1f135a40d","Type":"ContainerStarted","Data":"5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c"} Jun 03 16:25:43.864703 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:43.864303 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:43.890379 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:43.890315 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" podStartSLOduration=1.8937349879999998 podStartE2EDuration="7.890294445s" podCreationTimestamp="2026-06-03 16:25:36 +0000 UTC" firstStartedPulling="2026-06-03 16:25:37.366654 +0000 UTC m=+501.661847019" lastFinishedPulling="2026-06-03 16:25:43.363213455 +0000 UTC m=+507.658406476" observedRunningTime="2026-06-03 16:25:43.887581139 +0000 UTC m=+508.182774181" watchObservedRunningTime="2026-06-03 16:25:43.890294445 +0000 UTC m=+508.185487489" Jun 03 16:25:54.869129 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:54.869042 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:56.035176 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.035144 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg"] Jun 03 16:25:56.038392 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.038371 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.051691 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.051665 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg"] Jun 03 16:25:56.143161 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.143126 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg"] Jun 03 16:25:56.143416 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:25:56.143393 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-zvdc8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" podUID="6695017c-db49-4a24-b28e-e6bb52310171" Jun 03 16:25:56.192555 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.192523 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x"] Jun 03 16:25:56.195705 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.195684 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:56.196770 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.196752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvdc8\" (UniqueName: \"kubernetes.io/projected/6695017c-db49-4a24-b28e-e6bb52310171-kube-api-access-zvdc8\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" (UID: \"6695017c-db49-4a24-b28e-e6bb52310171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.196896 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.196876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6695017c-db49-4a24-b28e-e6bb52310171-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" (UID: \"6695017c-db49-4a24-b28e-e6bb52310171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.208088 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.208060 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x"] Jun 03 16:25:56.298309 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.298209 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" (UID: \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:56.298309 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.298268 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6695017c-db49-4a24-b28e-e6bb52310171-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" (UID: \"6695017c-db49-4a24-b28e-e6bb52310171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.298539 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.298376 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zvdc8\" (UniqueName: \"kubernetes.io/projected/6695017c-db49-4a24-b28e-e6bb52310171-kube-api-access-zvdc8\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" (UID: \"6695017c-db49-4a24-b28e-e6bb52310171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.298539 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.298420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98dt9\" (UniqueName: \"kubernetes.io/projected/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-kube-api-access-98dt9\") pod \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" (UID: \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:56.298610 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.298584 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6695017c-db49-4a24-b28e-e6bb52310171-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" (UID: \"6695017c-db49-4a24-b28e-e6bb52310171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.308507 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.308475 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvdc8\" (UniqueName: \"kubernetes.io/projected/6695017c-db49-4a24-b28e-e6bb52310171-kube-api-access-zvdc8\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" (UID: \"6695017c-db49-4a24-b28e-e6bb52310171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.399477 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.399433 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" (UID: \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:56.399652 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.399515 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-98dt9\" (UniqueName: \"kubernetes.io/projected/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-kube-api-access-98dt9\") pod \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" (UID: \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:56.399838 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.399818 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" (UID: \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:56.416680 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.416651 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98dt9\" (UniqueName: \"kubernetes.io/projected/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-kube-api-access-98dt9\") pod \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" (UID: \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:56.505837 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.505795 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:56.645358 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.645323 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl"] Jun 03 16:25:56.645634 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.645585 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" containerName="manager" containerID="cri-o://5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c" gracePeriod=2 Jun 03 16:25:56.649839 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.649792 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x"] Jun 03 16:25:56.653010 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:25:56.652978 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd11fbe_5d59_47c5_83b7_0be52ba3b093.slice/crio-9d93cc32111142edd3ca6d0fa58d94571e2461b5c570dff871ee44ca1764c454 WatchSource:0}: Error finding container 9d93cc32111142edd3ca6d0fa58d94571e2461b5c570dff871ee44ca1764c454: Status 404 returned error can't find the container with id 9d93cc32111142edd3ca6d0fa58d94571e2461b5c570dff871ee44ca1764c454 Jun 03 16:25:56.668375 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.663013 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl"] Jun 03 16:25:56.673623 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.673592 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg"] Jun 03 16:25:56.676716 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.675537 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf"] Jun 03 16:25:56.676716 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.676180 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" containerName="manager" Jun 03 16:25:56.676716 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.676195 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" containerName="manager" Jun 03 16:25:56.676716 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.676311 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" containerName="manager" Jun 03 16:25:56.679683 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.679662 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg"] Jun 03 16:25:56.679814 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.679801 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:56.688307 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.688275 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf"] Jun 03 16:25:56.692530 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.692504 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x"] Jun 03 16:25:56.700898 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.700871 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x"] Jun 03 16:25:56.720723 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.720662 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.725879 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.725855 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n"] Jun 03 16:25:56.733285 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.733263 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:56.742589 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.742563 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n"] Jun 03 16:25:56.770326 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.770284 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.803222 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.803110 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgb6s\" (UniqueName: \"kubernetes.io/projected/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-kube-api-access-wgb6s\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-n9nsf\" (UID: \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:56.803364 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.803225 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-n9nsf\" (UID: \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:56.880304 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.880282 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:56.882844 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.882819 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.904183 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.904157 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wgb6s\" (UniqueName: \"kubernetes.io/projected/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-kube-api-access-wgb6s\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-n9nsf\" (UID: \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:56.904335 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.904200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f20be85-7543-41b2-8fb1-73e7d000f4d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cr57n\" (UID: \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:56.904335 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.904255 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-n9nsf\" (UID: \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:56.904335 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.904288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq54n\" (UniqueName: \"kubernetes.io/projected/7f20be85-7543-41b2-8fb1-73e7d000f4d6-kube-api-access-pq54n\") pod \"kuadrant-operator-controller-manager-84b657d985-cr57n\" (UID: \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:56.904761 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.904735 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-n9nsf\" (UID: \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:56.905812 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.905786 2563 generic.go:358] "Generic (PLEG): container finished" podID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" containerID="5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c" exitCode=0 Jun 03 16:25:56.905887 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.905827 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" Jun 03 16:25:56.905887 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.905879 2563 scope.go:117] "RemoveContainer" containerID="5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c" Jun 03 16:25:56.907721 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.907705 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.907826 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.907732 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" containerName="manager" containerID="cri-o://959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75" gracePeriod=2 Jun 03 16:25:56.908789 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.908760 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.912203 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.912182 2563 status_manager.go:895] "Failed to get status for pod" podUID="6695017c-db49-4a24-b28e-e6bb52310171" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.912386 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.912372 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:56.914039 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.914019 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.914262 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.914242 2563 scope.go:117] "RemoveContainer" containerID="5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c" Jun 03 16:25:56.914499 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:25:56.914482 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c\": container with ID starting with 5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c not found: ID does not exist" containerID="5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c" Jun 03 16:25:56.914547 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.914507 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c"} err="failed to get container status \"5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c\": rpc error: code = NotFound desc = could not find container \"5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c\": container with ID starting with 5415a92fc4f9d05055eb41e641b8ba2c10cbcfb8f7f7bd23c358eaf45bbf4a3c not found: ID does not exist" Jun 03 16:25:56.916285 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.916265 2563 status_manager.go:895] "Failed to get status for pod" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" err="pods \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.918389 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.918371 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.920089 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.920071 2563 status_manager.go:895] "Failed to get status for pod" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" err="pods \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.921731 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.921714 2563 status_manager.go:895] "Failed to get status for pod" podUID="6695017c-db49-4a24-b28e-e6bb52310171" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:56.926458 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:56.926440 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgb6s\" (UniqueName: \"kubernetes.io/projected/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-kube-api-access-wgb6s\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-n9nsf\" (UID: \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:57.005485 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.005447 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6695017c-db49-4a24-b28e-e6bb52310171-extensions-socket-volume\") pod \"6695017c-db49-4a24-b28e-e6bb52310171\" (UID: \"6695017c-db49-4a24-b28e-e6bb52310171\") " Jun 03 16:25:57.005485 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.005493 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db1b88a7-2302-436d-a3dc-a3f1f135a40d-extensions-socket-volume\") pod \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\" (UID: \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\") " Jun 03 16:25:57.005732 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.005544 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgxpw\" (UniqueName: \"kubernetes.io/projected/db1b88a7-2302-436d-a3dc-a3f1f135a40d-kube-api-access-kgxpw\") pod \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\" (UID: \"db1b88a7-2302-436d-a3dc-a3f1f135a40d\") " Jun 03 16:25:57.005732 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.005594 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvdc8\" (UniqueName: \"kubernetes.io/projected/6695017c-db49-4a24-b28e-e6bb52310171-kube-api-access-zvdc8\") pod \"6695017c-db49-4a24-b28e-e6bb52310171\" (UID: \"6695017c-db49-4a24-b28e-e6bb52310171\") " Jun 03 16:25:57.005732 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.005695 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pq54n\" (UniqueName: \"kubernetes.io/projected/7f20be85-7543-41b2-8fb1-73e7d000f4d6-kube-api-access-pq54n\") pod \"kuadrant-operator-controller-manager-84b657d985-cr57n\" (UID: \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:57.005881 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.005736 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f20be85-7543-41b2-8fb1-73e7d000f4d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cr57n\" (UID: \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:57.005881 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.005787 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6695017c-db49-4a24-b28e-e6bb52310171-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6695017c-db49-4a24-b28e-e6bb52310171" (UID: "6695017c-db49-4a24-b28e-e6bb52310171"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:57.006132 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.006110 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f20be85-7543-41b2-8fb1-73e7d000f4d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cr57n\" (UID: \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:57.006310 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.006270 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1b88a7-2302-436d-a3dc-a3f1f135a40d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "db1b88a7-2302-436d-a3dc-a3f1f135a40d" (UID: "db1b88a7-2302-436d-a3dc-a3f1f135a40d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:57.007929 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.007905 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6695017c-db49-4a24-b28e-e6bb52310171-kube-api-access-zvdc8" (OuterVolumeSpecName: "kube-api-access-zvdc8") pod "6695017c-db49-4a24-b28e-e6bb52310171" (UID: "6695017c-db49-4a24-b28e-e6bb52310171"). InnerVolumeSpecName "kube-api-access-zvdc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:57.008067 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.008024 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1b88a7-2302-436d-a3dc-a3f1f135a40d-kube-api-access-kgxpw" (OuterVolumeSpecName: "kube-api-access-kgxpw") pod "db1b88a7-2302-436d-a3dc-a3f1f135a40d" (UID: "db1b88a7-2302-436d-a3dc-a3f1f135a40d"). InnerVolumeSpecName "kube-api-access-kgxpw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:57.014532 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.014507 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq54n\" (UniqueName: \"kubernetes.io/projected/7f20be85-7543-41b2-8fb1-73e7d000f4d6-kube-api-access-pq54n\") pod \"kuadrant-operator-controller-manager-84b657d985-cr57n\" (UID: \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:57.028903 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.028870 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:57.043835 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.043804 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:57.108734 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.107072 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvdc8\" (UniqueName: \"kubernetes.io/projected/6695017c-db49-4a24-b28e-e6bb52310171-kube-api-access-zvdc8\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:25:57.108734 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.107113 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6695017c-db49-4a24-b28e-e6bb52310171-extensions-socket-volume\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:25:57.108734 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.107128 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db1b88a7-2302-436d-a3dc-a3f1f135a40d-extensions-socket-volume\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:25:57.108734 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.107142 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgxpw\" (UniqueName: \"kubernetes.io/projected/db1b88a7-2302-436d-a3dc-a3f1f135a40d-kube-api-access-kgxpw\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:25:57.168099 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.168072 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:57.170311 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.170277 2563 status_manager.go:895] "Failed to get status for pod" podUID="6695017c-db49-4a24-b28e-e6bb52310171" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.172032 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.171981 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.173648 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.173611 2563 status_manager.go:895] "Failed to get status for pod" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" err="pods \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.209761 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.209733 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf"] Jun 03 16:25:57.211114 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:25:57.211086 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b0a7f7_7266_41c3_abbf_8c5f6d0c4a84.slice/crio-0efb50078f0ca384dca81647d6b659c0438432fb21734d193f8c6e51480fe773 WatchSource:0}: Error finding container 0efb50078f0ca384dca81647d6b659c0438432fb21734d193f8c6e51480fe773: Status 404 returned error can't find the container with id 0efb50078f0ca384dca81647d6b659c0438432fb21734d193f8c6e51480fe773 Jun 03 16:25:57.222122 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.222099 2563 status_manager.go:895] "Failed to get status for pod" podUID="6695017c-db49-4a24-b28e-e6bb52310171" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.223861 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.223834 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.225734 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.225716 2563 status_manager.go:895] "Failed to get status for pod" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" err="pods \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.234978 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.234940 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n"] Jun 03 16:25:57.240898 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:25:57.240876 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f20be85_7543_41b2_8fb1_73e7d000f4d6.slice/crio-ce70dc23d0c88f74239b813fd2e7673960116b22c3190eb55d7272bf10fb99d7 WatchSource:0}: Error finding container ce70dc23d0c88f74239b813fd2e7673960116b22c3190eb55d7272bf10fb99d7: Status 404 returned error can't find the container with id ce70dc23d0c88f74239b813fd2e7673960116b22c3190eb55d7272bf10fb99d7 Jun 03 16:25:57.308580 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.308556 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-extensions-socket-volume\") pod \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\" (UID: \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\") " Jun 03 16:25:57.308700 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.308608 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98dt9\" (UniqueName: \"kubernetes.io/projected/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-kube-api-access-98dt9\") pod \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\" (UID: \"9dd11fbe-5d59-47c5-83b7-0be52ba3b093\") " Jun 03 16:25:57.308817 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.308792 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9dd11fbe-5d59-47c5-83b7-0be52ba3b093" (UID: "9dd11fbe-5d59-47c5-83b7-0be52ba3b093"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:57.310756 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.310724 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-kube-api-access-98dt9" (OuterVolumeSpecName: "kube-api-access-98dt9") pod "9dd11fbe-5d59-47c5-83b7-0be52ba3b093" (UID: "9dd11fbe-5d59-47c5-83b7-0be52ba3b093"). InnerVolumeSpecName "kube-api-access-98dt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:57.409240 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.409202 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-extensions-socket-volume\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:25:57.409240 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.409233 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98dt9\" (UniqueName: \"kubernetes.io/projected/9dd11fbe-5d59-47c5-83b7-0be52ba3b093-kube-api-access-98dt9\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:25:57.913217 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.913175 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" event={"ID":"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84","Type":"ContainerStarted","Data":"f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a"} Jun 03 16:25:57.913217 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.913216 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" event={"ID":"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84","Type":"ContainerStarted","Data":"0efb50078f0ca384dca81647d6b659c0438432fb21734d193f8c6e51480fe773"} Jun 03 16:25:57.913433 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.913279 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:25:57.914282 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.914255 2563 generic.go:358] "Generic (PLEG): container finished" podID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" containerID="959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75" exitCode=2 Jun 03 16:25:57.914411 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.914292 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" Jun 03 16:25:57.914411 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.914320 2563 scope.go:117] "RemoveContainer" containerID="959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75" Jun 03 16:25:57.915659 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.915623 2563 status_manager.go:895] "Failed to get status for pod" podUID="6695017c-db49-4a24-b28e-e6bb52310171" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.915971 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.915937 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" Jun 03 16:25:57.916092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.915972 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" event={"ID":"7f20be85-7543-41b2-8fb1-73e7d000f4d6","Type":"ContainerStarted","Data":"3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318"} Jun 03 16:25:57.916092 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.916000 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" event={"ID":"7f20be85-7543-41b2-8fb1-73e7d000f4d6","Type":"ContainerStarted","Data":"ce70dc23d0c88f74239b813fd2e7673960116b22c3190eb55d7272bf10fb99d7"} Jun 03 16:25:57.916197 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.916162 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:25:57.923086 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.923067 2563 scope.go:117] "RemoveContainer" containerID="959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75" Jun 03 16:25:57.923412 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:25:57.923389 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75\": container with ID starting with 959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75 not found: ID does not exist" containerID="959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75" Jun 03 16:25:57.923479 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.923425 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75"} err="failed to get container status \"959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75\": rpc error: code = NotFound desc = could not find container \"959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75\": container with ID starting with 959ac045ae2fe8f012b1cb85eee3128f2dc3d6dbe3d9e60273678d503bb8ba75 not found: ID does not exist" Jun 03 16:25:57.941189 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.941139 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" podStartSLOduration=1.941122835 podStartE2EDuration="1.941122835s" podCreationTimestamp="2026-06-03 16:25:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:25:57.939663711 +0000 UTC m=+522.234856753" watchObservedRunningTime="2026-06-03 16:25:57.941122835 +0000 UTC m=+522.236315876" Jun 03 16:25:57.941926 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.941892 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.943898 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.943869 2563 status_manager.go:895] "Failed to get status for pod" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" err="pods \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.945928 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.945898 2563 status_manager.go:895] "Failed to get status for pod" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cgtsl" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-cgtsl\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.948052 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.948000 2563 status_manager.go:895] "Failed to get status for pod" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-g2k5x" err="pods \"kuadrant-operator-controller-manager-84b657d985-g2k5x\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:57.969971 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.969894 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" podStartSLOduration=1.96988108 podStartE2EDuration="1.96988108s" podCreationTimestamp="2026-06-03 16:25:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:25:57.968868681 +0000 UTC m=+522.264061721" watchObservedRunningTime="2026-06-03 16:25:57.96988108 +0000 UTC m=+522.265074122" Jun 03 16:25:57.971143 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:57.971110 2563 status_manager.go:895] "Failed to get status for pod" podUID="6695017c-db49-4a24-b28e-e6bb52310171" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-8cpxg\" is forbidden: User \"system:node:ip-10-0-139-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-61.ec2.internal' and this object" Jun 03 16:25:58.319199 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:58.319120 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6695017c-db49-4a24-b28e-e6bb52310171" path="/var/lib/kubelet/pods/6695017c-db49-4a24-b28e-e6bb52310171/volumes" Jun 03 16:25:58.319546 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:58.319351 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" path="/var/lib/kubelet/pods/9dd11fbe-5d59-47c5-83b7-0be52ba3b093/volumes" Jun 03 16:25:58.319646 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:25:58.319634 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db1b88a7-2302-436d-a3dc-a3f1f135a40d" path="/var/lib/kubelet/pods/db1b88a7-2302-436d-a3dc-a3f1f135a40d/volumes" Jun 03 16:26:08.922250 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:08.922216 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:26:08.922750 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:08.922645 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:26:09.002435 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.002406 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf"] Jun 03 16:26:09.002632 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.002611 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" podUID="16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84" containerName="manager" containerID="cri-o://f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a" gracePeriod=10 Jun 03 16:26:09.198216 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.198179 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2"] Jun 03 16:26:09.198546 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.198530 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" containerName="manager" Jun 03 16:26:09.198546 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.198546 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" containerName="manager" Jun 03 16:26:09.198925 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.198612 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="9dd11fbe-5d59-47c5-83b7-0be52ba3b093" containerName="manager" Jun 03 16:26:09.201532 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.201515 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.229463 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.229427 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2"] Jun 03 16:26:09.252495 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.252473 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:26:09.311584 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.311538 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a177361-44ee-4311-a1aa-adb6b54b26ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-w8hh2\" (UID: \"8a177361-44ee-4311-a1aa-adb6b54b26ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.311751 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.311635 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw5mh\" (UniqueName: \"kubernetes.io/projected/8a177361-44ee-4311-a1aa-adb6b54b26ad-kube-api-access-pw5mh\") pod \"kuadrant-operator-controller-manager-55c7f4c975-w8hh2\" (UID: \"8a177361-44ee-4311-a1aa-adb6b54b26ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.412960 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.412908 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-extensions-socket-volume\") pod \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\" (UID: \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\") " Jun 03 16:26:09.413151 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.413003 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgb6s\" (UniqueName: \"kubernetes.io/projected/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-kube-api-access-wgb6s\") pod \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\" (UID: \"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84\") " Jun 03 16:26:09.413151 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.413143 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a177361-44ee-4311-a1aa-adb6b54b26ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-w8hh2\" (UID: \"8a177361-44ee-4311-a1aa-adb6b54b26ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.413276 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.413178 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pw5mh\" (UniqueName: \"kubernetes.io/projected/8a177361-44ee-4311-a1aa-adb6b54b26ad-kube-api-access-pw5mh\") pod \"kuadrant-operator-controller-manager-55c7f4c975-w8hh2\" (UID: \"8a177361-44ee-4311-a1aa-adb6b54b26ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.413276 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.413258 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84" (UID: "16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:26:09.413507 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.413488 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a177361-44ee-4311-a1aa-adb6b54b26ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-w8hh2\" (UID: \"8a177361-44ee-4311-a1aa-adb6b54b26ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.415225 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.415203 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-kube-api-access-wgb6s" (OuterVolumeSpecName: "kube-api-access-wgb6s") pod "16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84" (UID: "16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84"). InnerVolumeSpecName "kube-api-access-wgb6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:26:09.437399 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.437326 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw5mh\" (UniqueName: \"kubernetes.io/projected/8a177361-44ee-4311-a1aa-adb6b54b26ad-kube-api-access-pw5mh\") pod \"kuadrant-operator-controller-manager-55c7f4c975-w8hh2\" (UID: \"8a177361-44ee-4311-a1aa-adb6b54b26ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.511640 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.511609 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.513702 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.513676 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-extensions-socket-volume\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:26:09.513758 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.513711 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgb6s\" (UniqueName: \"kubernetes.io/projected/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84-kube-api-access-wgb6s\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:26:09.653149 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.653106 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2"] Jun 03 16:26:09.657785 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:26:09.657754 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a177361_44ee_4311_a1aa_adb6b54b26ad.slice/crio-aad34152cca0d9c7ce56e4c2465cc512f7de38f084a176923a78eda8201364b0 WatchSource:0}: Error finding container aad34152cca0d9c7ce56e4c2465cc512f7de38f084a176923a78eda8201364b0: Status 404 returned error can't find the container with id aad34152cca0d9c7ce56e4c2465cc512f7de38f084a176923a78eda8201364b0 Jun 03 16:26:09.958124 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.958040 2563 generic.go:358] "Generic (PLEG): container finished" podID="16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84" containerID="f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a" exitCode=0 Jun 03 16:26:09.958124 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.958111 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" Jun 03 16:26:09.958602 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.958111 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" event={"ID":"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84","Type":"ContainerDied","Data":"f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a"} Jun 03 16:26:09.958602 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.958228 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf" event={"ID":"16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84","Type":"ContainerDied","Data":"0efb50078f0ca384dca81647d6b659c0438432fb21734d193f8c6e51480fe773"} Jun 03 16:26:09.958602 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.958246 2563 scope.go:117] "RemoveContainer" containerID="f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a" Jun 03 16:26:09.959708 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.959678 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" event={"ID":"8a177361-44ee-4311-a1aa-adb6b54b26ad","Type":"ContainerStarted","Data":"7c89e2d60e98f54b5a0e4805387c273057781f69e6103e10844ca7bd91700f3f"} Jun 03 16:26:09.959819 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.959717 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" event={"ID":"8a177361-44ee-4311-a1aa-adb6b54b26ad","Type":"ContainerStarted","Data":"aad34152cca0d9c7ce56e4c2465cc512f7de38f084a176923a78eda8201364b0"} Jun 03 16:26:09.959819 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.959756 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:09.966892 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.966877 2563 scope.go:117] "RemoveContainer" containerID="f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a" Jun 03 16:26:09.967172 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:26:09.967155 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a\": container with ID starting with f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a not found: ID does not exist" containerID="f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a" Jun 03 16:26:09.967227 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.967184 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a"} err="failed to get container status \"f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a\": rpc error: code = NotFound desc = could not find container \"f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a\": container with ID starting with f56e9dc7e197f9e7fbb77f564a2b1f1876e8aaef63389ac5bdb3b07413ea334a not found: ID does not exist" Jun 03 16:26:09.989128 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:09.989074 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" podStartSLOduration=0.989059303 podStartE2EDuration="989.059303ms" podCreationTimestamp="2026-06-03 16:26:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:26:09.987760393 +0000 UTC m=+534.282953431" watchObservedRunningTime="2026-06-03 16:26:09.989059303 +0000 UTC m=+534.284252343" Jun 03 16:26:10.003644 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:10.003612 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf"] Jun 03 16:26:10.008824 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:10.008799 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-n9nsf"] Jun 03 16:26:10.321785 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:10.321702 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84" path="/var/lib/kubelet/pods/16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84/volumes" Jun 03 16:26:20.965500 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:20.965467 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-w8hh2" Jun 03 16:26:21.023315 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.023284 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n"] Jun 03 16:26:21.023543 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.023505 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" podUID="7f20be85-7543-41b2-8fb1-73e7d000f4d6" containerName="manager" containerID="cri-o://3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318" gracePeriod=10 Jun 03 16:26:21.264890 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.264864 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:26:21.306959 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.306924 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f20be85-7543-41b2-8fb1-73e7d000f4d6-extensions-socket-volume\") pod \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\" (UID: \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\") " Jun 03 16:26:21.307109 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.306985 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq54n\" (UniqueName: \"kubernetes.io/projected/7f20be85-7543-41b2-8fb1-73e7d000f4d6-kube-api-access-pq54n\") pod \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\" (UID: \"7f20be85-7543-41b2-8fb1-73e7d000f4d6\") " Jun 03 16:26:21.307297 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.307275 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f20be85-7543-41b2-8fb1-73e7d000f4d6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7f20be85-7543-41b2-8fb1-73e7d000f4d6" (UID: "7f20be85-7543-41b2-8fb1-73e7d000f4d6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:26:21.309268 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.309240 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f20be85-7543-41b2-8fb1-73e7d000f4d6-kube-api-access-pq54n" (OuterVolumeSpecName: "kube-api-access-pq54n") pod "7f20be85-7543-41b2-8fb1-73e7d000f4d6" (UID: "7f20be85-7543-41b2-8fb1-73e7d000f4d6"). InnerVolumeSpecName "kube-api-access-pq54n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:26:21.407787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.407746 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f20be85-7543-41b2-8fb1-73e7d000f4d6-extensions-socket-volume\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:26:21.407787 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.407783 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq54n\" (UniqueName: \"kubernetes.io/projected/7f20be85-7543-41b2-8fb1-73e7d000f4d6-kube-api-access-pq54n\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:26:21.998121 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.998084 2563 generic.go:358] "Generic (PLEG): container finished" podID="7f20be85-7543-41b2-8fb1-73e7d000f4d6" containerID="3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318" exitCode=0 Jun 03 16:26:21.998550 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.998141 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" Jun 03 16:26:21.998550 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.998152 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" event={"ID":"7f20be85-7543-41b2-8fb1-73e7d000f4d6","Type":"ContainerDied","Data":"3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318"} Jun 03 16:26:21.998550 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.998179 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n" event={"ID":"7f20be85-7543-41b2-8fb1-73e7d000f4d6","Type":"ContainerDied","Data":"ce70dc23d0c88f74239b813fd2e7673960116b22c3190eb55d7272bf10fb99d7"} Jun 03 16:26:21.998550 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:21.998195 2563 scope.go:117] "RemoveContainer" containerID="3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318" Jun 03 16:26:22.006510 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:22.006495 2563 scope.go:117] "RemoveContainer" containerID="3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318" Jun 03 16:26:22.006790 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:26:22.006765 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318\": container with ID starting with 3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318 not found: ID does not exist" containerID="3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318" Jun 03 16:26:22.006886 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:22.006796 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318"} err="failed to get container status \"3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318\": rpc error: code = NotFound desc = could not find container \"3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318\": container with ID starting with 3fb1a88f9f21e4d3ac9aed32063db5b06773d274ac24569fa19a4626d8021318 not found: ID does not exist" Jun 03 16:26:22.020487 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:22.020461 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n"] Jun 03 16:26:22.025651 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:22.025623 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cr57n"] Jun 03 16:26:22.316897 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:22.316818 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f20be85-7543-41b2-8fb1-73e7d000f4d6" path="/var/lib/kubelet/pods/7f20be85-7543-41b2-8fb1-73e7d000f4d6/volumes" Jun 03 16:26:38.780599 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.780561 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:26:38.780984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.780884 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f20be85-7543-41b2-8fb1-73e7d000f4d6" containerName="manager" Jun 03 16:26:38.780984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.780895 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f20be85-7543-41b2-8fb1-73e7d000f4d6" containerName="manager" Jun 03 16:26:38.780984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.780905 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84" containerName="manager" Jun 03 16:26:38.780984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.780911 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84" containerName="manager" Jun 03 16:26:38.780984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.780973 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="16b0a7f7-7266-41c3-abbf-8c5f6d0c4a84" containerName="manager" Jun 03 16:26:38.780984 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.780984 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="7f20be85-7543-41b2-8fb1-73e7d000f4d6" containerName="manager" Jun 03 16:26:38.783678 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.783654 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:38.786277 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.786254 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 03 16:26:38.786567 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.786539 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5t7wf\"" Jun 03 16:26:38.795106 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.795085 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:26:38.823268 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.823231 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:26:38.859168 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.859111 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/889de063-6ecd-4af4-9454-a87b9bea77dc-config-file\") pod \"limitador-limitador-78c99df468-9tcgm\" (UID: \"889de063-6ecd-4af4-9454-a87b9bea77dc\") " pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:38.859350 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.859212 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f46ng\" (UniqueName: \"kubernetes.io/projected/889de063-6ecd-4af4-9454-a87b9bea77dc-kube-api-access-f46ng\") pod \"limitador-limitador-78c99df468-9tcgm\" (UID: \"889de063-6ecd-4af4-9454-a87b9bea77dc\") " pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:38.959996 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.959930 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/889de063-6ecd-4af4-9454-a87b9bea77dc-config-file\") pod \"limitador-limitador-78c99df468-9tcgm\" (UID: \"889de063-6ecd-4af4-9454-a87b9bea77dc\") " pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:38.959996 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.960002 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f46ng\" (UniqueName: \"kubernetes.io/projected/889de063-6ecd-4af4-9454-a87b9bea77dc-kube-api-access-f46ng\") pod \"limitador-limitador-78c99df468-9tcgm\" (UID: \"889de063-6ecd-4af4-9454-a87b9bea77dc\") " pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:38.960552 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.960530 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/889de063-6ecd-4af4-9454-a87b9bea77dc-config-file\") pod \"limitador-limitador-78c99df468-9tcgm\" (UID: \"889de063-6ecd-4af4-9454-a87b9bea77dc\") " pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:38.968390 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:38.968353 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46ng\" (UniqueName: \"kubernetes.io/projected/889de063-6ecd-4af4-9454-a87b9bea77dc-kube-api-access-f46ng\") pod \"limitador-limitador-78c99df468-9tcgm\" (UID: \"889de063-6ecd-4af4-9454-a87b9bea77dc\") " pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:39.094860 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.094769 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:39.235048 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.234964 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:26:39.237786 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:26:39.237756 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889de063_6ecd_4af4_9454_a87b9bea77dc.slice/crio-030a85291739166cc584198388b2e9ab5d58455c6ccdc363b7d3e5f57141121f WatchSource:0}: Error finding container 030a85291739166cc584198388b2e9ab5d58455c6ccdc363b7d3e5f57141121f: Status 404 returned error can't find the container with id 030a85291739166cc584198388b2e9ab5d58455c6ccdc363b7d3e5f57141121f Jun 03 16:26:39.444500 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.444468 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-vng2f"] Jun 03 16:26:39.448827 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.448795 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-vng2f" Jun 03 16:26:39.451683 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.451662 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-f2t4m\"" Jun 03 16:26:39.452843 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.452819 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-vng2f"] Jun 03 16:26:39.565004 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.564969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn4fq\" (UniqueName: \"kubernetes.io/projected/28bf5391-36e9-41db-852e-96c0e081ab6c-kube-api-access-bn4fq\") pod \"authorino-7498df8756-vng2f\" (UID: \"28bf5391-36e9-41db-852e-96c0e081ab6c\") " pod="kuadrant-system/authorino-7498df8756-vng2f" Jun 03 16:26:39.665745 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.665706 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bn4fq\" (UniqueName: \"kubernetes.io/projected/28bf5391-36e9-41db-852e-96c0e081ab6c-kube-api-access-bn4fq\") pod \"authorino-7498df8756-vng2f\" (UID: \"28bf5391-36e9-41db-852e-96c0e081ab6c\") " pod="kuadrant-system/authorino-7498df8756-vng2f" Jun 03 16:26:39.674492 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.674466 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn4fq\" (UniqueName: \"kubernetes.io/projected/28bf5391-36e9-41db-852e-96c0e081ab6c-kube-api-access-bn4fq\") pod \"authorino-7498df8756-vng2f\" (UID: \"28bf5391-36e9-41db-852e-96c0e081ab6c\") " pod="kuadrant-system/authorino-7498df8756-vng2f" Jun 03 16:26:39.765135 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.765040 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-vng2f" Jun 03 16:26:39.937793 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:39.937732 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-vng2f"] Jun 03 16:26:39.942685 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:26:39.942648 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28bf5391_36e9_41db_852e_96c0e081ab6c.slice/crio-d2cb75b3c3c35e98c9ec0a1ae10aa79cbdde184466aeed444a734363fe3c31b5 WatchSource:0}: Error finding container d2cb75b3c3c35e98c9ec0a1ae10aa79cbdde184466aeed444a734363fe3c31b5: Status 404 returned error can't find the container with id d2cb75b3c3c35e98c9ec0a1ae10aa79cbdde184466aeed444a734363fe3c31b5 Jun 03 16:26:40.056863 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:40.056769 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-vng2f" event={"ID":"28bf5391-36e9-41db-852e-96c0e081ab6c","Type":"ContainerStarted","Data":"d2cb75b3c3c35e98c9ec0a1ae10aa79cbdde184466aeed444a734363fe3c31b5"} Jun 03 16:26:40.058190 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:40.058159 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" event={"ID":"889de063-6ecd-4af4-9454-a87b9bea77dc","Type":"ContainerStarted","Data":"030a85291739166cc584198388b2e9ab5d58455c6ccdc363b7d3e5f57141121f"} Jun 03 16:26:44.075681 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:44.075575 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-vng2f" event={"ID":"28bf5391-36e9-41db-852e-96c0e081ab6c","Type":"ContainerStarted","Data":"3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab"} Jun 03 16:26:44.077002 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:44.076976 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" event={"ID":"889de063-6ecd-4af4-9454-a87b9bea77dc","Type":"ContainerStarted","Data":"b352338439c71290ea1eb1095eb712d578ad8679f30a3ee69ee12f844270128c"} Jun 03 16:26:44.077155 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:44.077068 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:26:44.091611 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:44.091564 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-vng2f" podStartSLOduration=1.284939707 podStartE2EDuration="5.091550062s" podCreationTimestamp="2026-06-03 16:26:39 +0000 UTC" firstStartedPulling="2026-06-03 16:26:39.944377794 +0000 UTC m=+564.239570816" lastFinishedPulling="2026-06-03 16:26:43.750988152 +0000 UTC m=+568.046181171" observedRunningTime="2026-06-03 16:26:44.089291844 +0000 UTC m=+568.384484885" watchObservedRunningTime="2026-06-03 16:26:44.091550062 +0000 UTC m=+568.386743102" Jun 03 16:26:44.106000 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:44.105920 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" podStartSLOduration=1.646322485 podStartE2EDuration="6.105899732s" podCreationTimestamp="2026-06-03 16:26:38 +0000 UTC" firstStartedPulling="2026-06-03 16:26:39.239492656 +0000 UTC m=+563.534685674" lastFinishedPulling="2026-06-03 16:26:43.699069898 +0000 UTC m=+567.994262921" observedRunningTime="2026-06-03 16:26:44.104745012 +0000 UTC m=+568.399938055" watchObservedRunningTime="2026-06-03 16:26:44.105899732 +0000 UTC m=+568.401092773" Jun 03 16:26:55.082049 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:26:55.082008 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-9tcgm" Jun 03 16:27:16.207703 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:16.207673 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:27:16.208864 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:16.208838 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:27:19.734759 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:19.734725 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-rkjzw"] Jun 03 16:27:19.741560 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:19.741538 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" Jun 03 16:27:19.746997 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:19.746963 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-rkjzw"] Jun 03 16:27:19.814354 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:19.814318 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkphh\" (UniqueName: \"kubernetes.io/projected/28196d60-c3ea-43d1-b2e6-1d5e1eb03b35-kube-api-access-hkphh\") pod \"authorino-8b475cf9f-rkjzw\" (UID: \"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35\") " pod="kuadrant-system/authorino-8b475cf9f-rkjzw" Jun 03 16:27:19.915622 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:19.915586 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hkphh\" (UniqueName: \"kubernetes.io/projected/28196d60-c3ea-43d1-b2e6-1d5e1eb03b35-kube-api-access-hkphh\") pod \"authorino-8b475cf9f-rkjzw\" (UID: \"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35\") " pod="kuadrant-system/authorino-8b475cf9f-rkjzw" Jun 03 16:27:19.923708 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:19.923682 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkphh\" (UniqueName: \"kubernetes.io/projected/28196d60-c3ea-43d1-b2e6-1d5e1eb03b35-kube-api-access-hkphh\") pod \"authorino-8b475cf9f-rkjzw\" (UID: \"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35\") " pod="kuadrant-system/authorino-8b475cf9f-rkjzw" Jun 03 16:27:19.989772 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:19.989687 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-rkjzw"] Jun 03 16:27:19.989917 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:19.989911 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" Jun 03 16:27:20.015373 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.015341 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cbb7c7c49-ng96c"] Jun 03 16:27:20.023399 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.023377 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cbb7c7c49-ng96c" Jun 03 16:27:20.028023 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.027985 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cbb7c7c49-ng96c"] Jun 03 16:27:20.117846 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.117813 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw9gm\" (UniqueName: \"kubernetes.io/projected/ae961a74-cf88-4c10-8157-cd88670f4274-kube-api-access-xw9gm\") pod \"authorino-cbb7c7c49-ng96c\" (UID: \"ae961a74-cf88-4c10-8157-cd88670f4274\") " pod="kuadrant-system/authorino-cbb7c7c49-ng96c" Jun 03 16:27:20.124342 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.124312 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-rkjzw"] Jun 03 16:27:20.128790 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:27:20.128760 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28196d60_c3ea_43d1_b2e6_1d5e1eb03b35.slice/crio-4c2dd4c9af7e43e60caea9f1c75e01f1194976396fc2ee44bce6ad7f5c8121d5 WatchSource:0}: Error finding container 4c2dd4c9af7e43e60caea9f1c75e01f1194976396fc2ee44bce6ad7f5c8121d5: Status 404 returned error can't find the container with id 4c2dd4c9af7e43e60caea9f1c75e01f1194976396fc2ee44bce6ad7f5c8121d5 Jun 03 16:27:20.191462 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.191427 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" event={"ID":"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35","Type":"ContainerStarted","Data":"4c2dd4c9af7e43e60caea9f1c75e01f1194976396fc2ee44bce6ad7f5c8121d5"} Jun 03 16:27:20.196340 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.196313 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cbb7c7c49-ng96c"] Jun 03 16:27:20.196614 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:27:20.196593 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xw9gm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-cbb7c7c49-ng96c" podUID="ae961a74-cf88-4c10-8157-cd88670f4274" Jun 03 16:27:20.218602 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.218566 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9gm\" (UniqueName: \"kubernetes.io/projected/ae961a74-cf88-4c10-8157-cd88670f4274-kube-api-access-xw9gm\") pod \"authorino-cbb7c7c49-ng96c\" (UID: \"ae961a74-cf88-4c10-8157-cd88670f4274\") " pod="kuadrant-system/authorino-cbb7c7c49-ng96c" Jun 03 16:27:20.223814 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.223782 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69596fbcfc-94zwp"] Jun 03 16:27:20.226197 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.226177 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69596fbcfc-94zwp" Jun 03 16:27:20.228224 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.228189 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw9gm\" (UniqueName: \"kubernetes.io/projected/ae961a74-cf88-4c10-8157-cd88670f4274-kube-api-access-xw9gm\") pod \"authorino-cbb7c7c49-ng96c\" (UID: \"ae961a74-cf88-4c10-8157-cd88670f4274\") " pod="kuadrant-system/authorino-cbb7c7c49-ng96c" Jun 03 16:27:20.228438 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.228422 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 03 16:27:20.236352 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.236330 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69596fbcfc-94zwp"] Jun 03 16:27:20.319235 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.319203 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns824\" (UniqueName: \"kubernetes.io/projected/3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e-kube-api-access-ns824\") pod \"authorino-69596fbcfc-94zwp\" (UID: \"3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e\") " pod="kuadrant-system/authorino-69596fbcfc-94zwp" Jun 03 16:27:20.319235 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.319240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e-tls-cert\") pod \"authorino-69596fbcfc-94zwp\" (UID: \"3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e\") " pod="kuadrant-system/authorino-69596fbcfc-94zwp" Jun 03 16:27:20.420581 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.420540 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ns824\" (UniqueName: \"kubernetes.io/projected/3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e-kube-api-access-ns824\") pod \"authorino-69596fbcfc-94zwp\" (UID: \"3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e\") " pod="kuadrant-system/authorino-69596fbcfc-94zwp" Jun 03 16:27:20.420770 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.420588 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e-tls-cert\") pod \"authorino-69596fbcfc-94zwp\" (UID: \"3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e\") " pod="kuadrant-system/authorino-69596fbcfc-94zwp" Jun 03 16:27:20.423149 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.423127 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e-tls-cert\") pod \"authorino-69596fbcfc-94zwp\" (UID: \"3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e\") " pod="kuadrant-system/authorino-69596fbcfc-94zwp" Jun 03 16:27:20.428008 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.427993 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns824\" (UniqueName: \"kubernetes.io/projected/3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e-kube-api-access-ns824\") pod \"authorino-69596fbcfc-94zwp\" (UID: \"3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e\") " pod="kuadrant-system/authorino-69596fbcfc-94zwp" Jun 03 16:27:20.544019 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.543978 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69596fbcfc-94zwp" Jun 03 16:27:20.672649 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:20.672616 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69596fbcfc-94zwp"] Jun 03 16:27:20.675607 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:27:20.675579 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d5ef6f6_bc37_4da1_a1e8_d49c1eb9e04e.slice/crio-0a8ad933eaebefe5994ff418b72e40208bf084ce1f4c35b2e4a4278af7695e5f WatchSource:0}: Error finding container 0a8ad933eaebefe5994ff418b72e40208bf084ce1f4c35b2e4a4278af7695e5f: Status 404 returned error can't find the container with id 0a8ad933eaebefe5994ff418b72e40208bf084ce1f4c35b2e4a4278af7695e5f Jun 03 16:27:21.196501 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.196462 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" event={"ID":"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35","Type":"ContainerStarted","Data":"fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343"} Jun 03 16:27:21.196969 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.196545 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" podUID="28196d60-c3ea-43d1-b2e6-1d5e1eb03b35" containerName="authorino" containerID="cri-o://fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343" gracePeriod=30 Jun 03 16:27:21.198357 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.198079 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69596fbcfc-94zwp" event={"ID":"3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e","Type":"ContainerStarted","Data":"7af9d9df2ecfe7e17bcfd26026ebc334ea5b04746c06341c74932795c689d741"} Jun 03 16:27:21.198357 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.198173 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69596fbcfc-94zwp" event={"ID":"3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e","Type":"ContainerStarted","Data":"0a8ad933eaebefe5994ff418b72e40208bf084ce1f4c35b2e4a4278af7695e5f"} Jun 03 16:27:21.198357 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.198296 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cbb7c7c49-ng96c" Jun 03 16:27:21.204021 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.203916 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cbb7c7c49-ng96c" Jun 03 16:27:21.210913 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.210767 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" podStartSLOduration=1.861855716 podStartE2EDuration="2.210653978s" podCreationTimestamp="2026-06-03 16:27:19 +0000 UTC" firstStartedPulling="2026-06-03 16:27:20.130079889 +0000 UTC m=+604.425272912" lastFinishedPulling="2026-06-03 16:27:20.478878154 +0000 UTC m=+604.774071174" observedRunningTime="2026-06-03 16:27:21.210408152 +0000 UTC m=+605.505601196" watchObservedRunningTime="2026-06-03 16:27:21.210653978 +0000 UTC m=+605.505847022" Jun 03 16:27:21.232096 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.232039 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-69596fbcfc-94zwp" podStartSLOduration=0.85194419 podStartE2EDuration="1.232022722s" podCreationTimestamp="2026-06-03 16:27:20 +0000 UTC" firstStartedPulling="2026-06-03 16:27:20.6770093 +0000 UTC m=+604.972202320" lastFinishedPulling="2026-06-03 16:27:21.057087833 +0000 UTC m=+605.352280852" observedRunningTime="2026-06-03 16:27:21.231188212 +0000 UTC m=+605.526381254" watchObservedRunningTime="2026-06-03 16:27:21.232022722 +0000 UTC m=+605.527215916" Jun 03 16:27:21.260888 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.260855 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-vng2f"] Jun 03 16:27:21.261117 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.261094 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-vng2f" podUID="28bf5391-36e9-41db-852e-96c0e081ab6c" containerName="authorino" containerID="cri-o://3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab" gracePeriod=30 Jun 03 16:27:21.328061 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.327932 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw9gm\" (UniqueName: \"kubernetes.io/projected/ae961a74-cf88-4c10-8157-cd88670f4274-kube-api-access-xw9gm\") pod \"ae961a74-cf88-4c10-8157-cd88670f4274\" (UID: \"ae961a74-cf88-4c10-8157-cd88670f4274\") " Jun 03 16:27:21.329996 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.329970 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae961a74-cf88-4c10-8157-cd88670f4274-kube-api-access-xw9gm" (OuterVolumeSpecName: "kube-api-access-xw9gm") pod "ae961a74-cf88-4c10-8157-cd88670f4274" (UID: "ae961a74-cf88-4c10-8157-cd88670f4274"). InnerVolumeSpecName "kube-api-access-xw9gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:27:21.429140 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.429109 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw9gm\" (UniqueName: \"kubernetes.io/projected/ae961a74-cf88-4c10-8157-cd88670f4274-kube-api-access-xw9gm\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:27:21.492701 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.492675 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" Jun 03 16:27:21.528797 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.528773 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-vng2f" Jun 03 16:27:21.630768 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.630744 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn4fq\" (UniqueName: \"kubernetes.io/projected/28bf5391-36e9-41db-852e-96c0e081ab6c-kube-api-access-bn4fq\") pod \"28bf5391-36e9-41db-852e-96c0e081ab6c\" (UID: \"28bf5391-36e9-41db-852e-96c0e081ab6c\") " Jun 03 16:27:21.630924 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.630799 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkphh\" (UniqueName: \"kubernetes.io/projected/28196d60-c3ea-43d1-b2e6-1d5e1eb03b35-kube-api-access-hkphh\") pod \"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35\" (UID: \"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35\") " Jun 03 16:27:21.632974 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.632935 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28196d60-c3ea-43d1-b2e6-1d5e1eb03b35-kube-api-access-hkphh" (OuterVolumeSpecName: "kube-api-access-hkphh") pod "28196d60-c3ea-43d1-b2e6-1d5e1eb03b35" (UID: "28196d60-c3ea-43d1-b2e6-1d5e1eb03b35"). InnerVolumeSpecName "kube-api-access-hkphh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:27:21.632974 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.632938 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28bf5391-36e9-41db-852e-96c0e081ab6c-kube-api-access-bn4fq" (OuterVolumeSpecName: "kube-api-access-bn4fq") pod "28bf5391-36e9-41db-852e-96c0e081ab6c" (UID: "28bf5391-36e9-41db-852e-96c0e081ab6c"). InnerVolumeSpecName "kube-api-access-bn4fq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:27:21.731764 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.731730 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn4fq\" (UniqueName: \"kubernetes.io/projected/28bf5391-36e9-41db-852e-96c0e081ab6c-kube-api-access-bn4fq\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:27:21.731764 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:21.731760 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkphh\" (UniqueName: \"kubernetes.io/projected/28196d60-c3ea-43d1-b2e6-1d5e1eb03b35-kube-api-access-hkphh\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:27:22.202280 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.202191 2563 generic.go:358] "Generic (PLEG): container finished" podID="28196d60-c3ea-43d1-b2e6-1d5e1eb03b35" containerID="fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343" exitCode=0 Jun 03 16:27:22.202280 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.202247 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" Jun 03 16:27:22.202280 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.202272 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" event={"ID":"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35","Type":"ContainerDied","Data":"fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343"} Jun 03 16:27:22.202804 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.202311 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-rkjzw" event={"ID":"28196d60-c3ea-43d1-b2e6-1d5e1eb03b35","Type":"ContainerDied","Data":"4c2dd4c9af7e43e60caea9f1c75e01f1194976396fc2ee44bce6ad7f5c8121d5"} Jun 03 16:27:22.202804 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.202327 2563 scope.go:117] "RemoveContainer" containerID="fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343" Jun 03 16:27:22.203610 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.203588 2563 generic.go:358] "Generic (PLEG): container finished" podID="28bf5391-36e9-41db-852e-96c0e081ab6c" containerID="3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab" exitCode=0 Jun 03 16:27:22.203868 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.203630 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-vng2f" Jun 03 16:27:22.203868 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.203668 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-vng2f" event={"ID":"28bf5391-36e9-41db-852e-96c0e081ab6c","Type":"ContainerDied","Data":"3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab"} Jun 03 16:27:22.203868 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.203701 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-vng2f" event={"ID":"28bf5391-36e9-41db-852e-96c0e081ab6c","Type":"ContainerDied","Data":"d2cb75b3c3c35e98c9ec0a1ae10aa79cbdde184466aeed444a734363fe3c31b5"} Jun 03 16:27:22.203868 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.203735 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cbb7c7c49-ng96c" Jun 03 16:27:22.211745 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.211724 2563 scope.go:117] "RemoveContainer" containerID="fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343" Jun 03 16:27:22.212114 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:27:22.212079 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343\": container with ID starting with fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343 not found: ID does not exist" containerID="fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343" Jun 03 16:27:22.212175 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.212111 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343"} err="failed to get container status \"fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343\": rpc error: code = NotFound desc = could not find container \"fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343\": container with ID starting with fb4ca4d87e117e465bd8433b7afacaa2f8163871e3ea8c3830a82e09f6bd0343 not found: ID does not exist" Jun 03 16:27:22.212175 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.212135 2563 scope.go:117] "RemoveContainer" containerID="3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab" Jun 03 16:27:22.221116 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.221096 2563 scope.go:117] "RemoveContainer" containerID="3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab" Jun 03 16:27:22.221339 ip-10-0-139-61 kubenswrapper[2563]: E0603 16:27:22.221321 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab\": container with ID starting with 3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab not found: ID does not exist" containerID="3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab" Jun 03 16:27:22.221385 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.221347 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab"} err="failed to get container status \"3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab\": rpc error: code = NotFound desc = could not find container \"3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab\": container with ID starting with 3f42fcf8a9deb0e4c1426f28dceea15d981e69aacbf34bbcb9bca3b9e05e4bab not found: ID does not exist" Jun 03 16:27:22.252306 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.246176 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cbb7c7c49-ng96c"] Jun 03 16:27:22.254911 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.254879 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cbb7c7c49-ng96c"] Jun 03 16:27:22.269228 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.269203 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-vng2f"] Jun 03 16:27:22.275590 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.275564 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-vng2f"] Jun 03 16:27:22.292336 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.292311 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-rkjzw"] Jun 03 16:27:22.296236 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.296213 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-rkjzw"] Jun 03 16:27:22.317216 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.317190 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28196d60-c3ea-43d1-b2e6-1d5e1eb03b35" path="/var/lib/kubelet/pods/28196d60-c3ea-43d1-b2e6-1d5e1eb03b35/volumes" Jun 03 16:27:22.317498 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.317486 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28bf5391-36e9-41db-852e-96c0e081ab6c" path="/var/lib/kubelet/pods/28bf5391-36e9-41db-852e-96c0e081ab6c/volumes" Jun 03 16:27:22.317753 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:22.317741 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae961a74-cf88-4c10-8157-cd88670f4274" path="/var/lib/kubelet/pods/ae961a74-cf88-4c10-8157-cd88670f4274/volumes" Jun 03 16:27:28.858206 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:28.858175 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:27:59.601182 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:27:59.601137 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:28:11.603600 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:28:11.603557 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:28:16.219262 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:28:16.219225 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:28:18.711545 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:28:18.711501 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:28:24.702739 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:28:24.702700 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:28:37.804816 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:28:37.804774 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:28:43.704577 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:28:43.704535 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9tcgm"] Jun 03 16:30:00.138608 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.138571 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29675070-5tx49"] Jun 03 16:30:00.139041 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.138866 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bf5391-36e9-41db-852e-96c0e081ab6c" containerName="authorino" Jun 03 16:30:00.139041 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.138877 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bf5391-36e9-41db-852e-96c0e081ab6c" containerName="authorino" Jun 03 16:30:00.139041 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.138902 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28196d60-c3ea-43d1-b2e6-1d5e1eb03b35" containerName="authorino" Jun 03 16:30:00.139041 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.138909 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28196d60-c3ea-43d1-b2e6-1d5e1eb03b35" containerName="authorino" Jun 03 16:30:00.139041 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.138976 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="28196d60-c3ea-43d1-b2e6-1d5e1eb03b35" containerName="authorino" Jun 03 16:30:00.139041 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.138988 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="28bf5391-36e9-41db-852e-96c0e081ab6c" containerName="authorino" Jun 03 16:30:00.141907 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.141890 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" Jun 03 16:30:00.144271 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.144253 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-j8x4v\"" Jun 03 16:30:00.145434 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.145409 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlp5n\" (UniqueName: \"kubernetes.io/projected/9699b5cd-a58a-471a-86f5-8eee00d4e0a9-kube-api-access-jlp5n\") pod \"maas-api-key-cleanup-29675070-5tx49\" (UID: \"9699b5cd-a58a-471a-86f5-8eee00d4e0a9\") " pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" Jun 03 16:30:00.149099 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.149055 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29675070-5tx49"] Jun 03 16:30:00.246449 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.246414 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jlp5n\" (UniqueName: \"kubernetes.io/projected/9699b5cd-a58a-471a-86f5-8eee00d4e0a9-kube-api-access-jlp5n\") pod \"maas-api-key-cleanup-29675070-5tx49\" (UID: \"9699b5cd-a58a-471a-86f5-8eee00d4e0a9\") " pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" Jun 03 16:30:00.255179 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.255144 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlp5n\" (UniqueName: \"kubernetes.io/projected/9699b5cd-a58a-471a-86f5-8eee00d4e0a9-kube-api-access-jlp5n\") pod \"maas-api-key-cleanup-29675070-5tx49\" (UID: \"9699b5cd-a58a-471a-86f5-8eee00d4e0a9\") " pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" Jun 03 16:30:00.454210 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.454123 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" Jun 03 16:30:00.586075 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.586033 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29675070-5tx49"] Jun 03 16:30:00.588916 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:30:00.588891 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9699b5cd_a58a_471a_86f5_8eee00d4e0a9.slice/crio-ff8fe6ab1334066fe4b6dc95e00c9b4a5201560ae874d5c3959b9b2a74d528f3 WatchSource:0}: Error finding container ff8fe6ab1334066fe4b6dc95e00c9b4a5201560ae874d5c3959b9b2a74d528f3: Status 404 returned error can't find the container with id ff8fe6ab1334066fe4b6dc95e00c9b4a5201560ae874d5c3959b9b2a74d528f3 Jun 03 16:30:00.590636 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.590620 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:30:00.721993 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:00.721887 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" event={"ID":"9699b5cd-a58a-471a-86f5-8eee00d4e0a9","Type":"ContainerStarted","Data":"ff8fe6ab1334066fe4b6dc95e00c9b4a5201560ae874d5c3959b9b2a74d528f3"} Jun 03 16:30:14.772108 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:14.772064 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" event={"ID":"9699b5cd-a58a-471a-86f5-8eee00d4e0a9","Type":"ContainerStarted","Data":"adedec7e169dcf085ed233e308004b9bcdb1dd310402c05ccf3ad3874bea34bf"} Jun 03 16:30:14.791844 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:14.791796 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" podStartSLOduration=1.081369286 podStartE2EDuration="14.79178114s" podCreationTimestamp="2026-06-03 16:30:00 +0000 UTC" firstStartedPulling="2026-06-03 16:30:00.590743617 +0000 UTC m=+764.885936636" lastFinishedPulling="2026-06-03 16:30:14.301155468 +0000 UTC m=+778.596348490" observedRunningTime="2026-06-03 16:30:14.790861397 +0000 UTC m=+779.086054439" watchObservedRunningTime="2026-06-03 16:30:14.79178114 +0000 UTC m=+779.086974182" Jun 03 16:30:15.777136 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:15.777100 2563 generic.go:358] "Generic (PLEG): container finished" podID="9699b5cd-a58a-471a-86f5-8eee00d4e0a9" containerID="adedec7e169dcf085ed233e308004b9bcdb1dd310402c05ccf3ad3874bea34bf" exitCode=0 Jun 03 16:30:15.777613 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:15.777189 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" event={"ID":"9699b5cd-a58a-471a-86f5-8eee00d4e0a9","Type":"ContainerDied","Data":"adedec7e169dcf085ed233e308004b9bcdb1dd310402c05ccf3ad3874bea34bf"} Jun 03 16:30:16.914215 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:16.914192 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" Jun 03 16:30:16.989417 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:16.989375 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlp5n\" (UniqueName: \"kubernetes.io/projected/9699b5cd-a58a-471a-86f5-8eee00d4e0a9-kube-api-access-jlp5n\") pod \"9699b5cd-a58a-471a-86f5-8eee00d4e0a9\" (UID: \"9699b5cd-a58a-471a-86f5-8eee00d4e0a9\") " Jun 03 16:30:16.991646 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:16.991623 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9699b5cd-a58a-471a-86f5-8eee00d4e0a9-kube-api-access-jlp5n" (OuterVolumeSpecName: "kube-api-access-jlp5n") pod "9699b5cd-a58a-471a-86f5-8eee00d4e0a9" (UID: "9699b5cd-a58a-471a-86f5-8eee00d4e0a9"). InnerVolumeSpecName "kube-api-access-jlp5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:30:17.090588 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:17.090498 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlp5n\" (UniqueName: \"kubernetes.io/projected/9699b5cd-a58a-471a-86f5-8eee00d4e0a9-kube-api-access-jlp5n\") on node \"ip-10-0-139-61.ec2.internal\" DevicePath \"\"" Jun 03 16:30:17.784539 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:17.784506 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" event={"ID":"9699b5cd-a58a-471a-86f5-8eee00d4e0a9","Type":"ContainerDied","Data":"ff8fe6ab1334066fe4b6dc95e00c9b4a5201560ae874d5c3959b9b2a74d528f3"} Jun 03 16:30:17.784539 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:17.784540 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff8fe6ab1334066fe4b6dc95e00c9b4a5201560ae874d5c3959b9b2a74d528f3" Jun 03 16:30:17.784741 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:30:17.784558 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29675070-5tx49" Jun 03 16:32:16.229575 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:32:16.229494 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:32:16.231173 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:32:16.231148 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:33:43.149046 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:43.149009 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69596fbcfc-94zwp_3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e/authorino/0.log" Jun 03 16:33:47.825462 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:47.825437 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29675070-5tx49_9699b5cd-a58a-471a-86f5-8eee00d4e0a9/cleanup/0.log" Jun 03 16:33:48.353391 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:48.353360 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c656db94d-k6nfz_16dc22cf-d4e4-41e8-9beb-7b2f3217a46f/manager/0.log" Jun 03 16:33:49.902145 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:49.902115 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69596fbcfc-94zwp_3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e/authorino/0.log" Jun 03 16:33:50.488283 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:50.488252 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-w8hh2_8a177361-44ee-4311-a1aa-adb6b54b26ad/manager/0.log" Jun 03 16:33:50.607924 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:50.607896 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-9tcgm_889de063-6ecd-4af4-9454-a87b9bea77dc/limitador/0.log" Jun 03 16:33:51.362068 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:51.362034 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b4fb99578-2dwtm_91b33d16-f254-4940-8265-039e65e0059a/kube-auth-proxy/0.log" Jun 03 16:33:57.221890 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.221849 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-clkt5/must-gather-97xb6"] Jun 03 16:33:57.222297 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.222256 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9699b5cd-a58a-471a-86f5-8eee00d4e0a9" containerName="cleanup" Jun 03 16:33:57.222297 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.222268 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9699b5cd-a58a-471a-86f5-8eee00d4e0a9" containerName="cleanup" Jun 03 16:33:57.222368 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.222326 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="9699b5cd-a58a-471a-86f5-8eee00d4e0a9" containerName="cleanup" Jun 03 16:33:57.225348 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.225332 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-clkt5/must-gather-97xb6" Jun 03 16:33:57.227612 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.227590 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-clkt5\"/\"kube-root-ca.crt\"" Jun 03 16:33:57.227744 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.227725 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-clkt5\"/\"default-dockercfg-zpvg7\"" Jun 03 16:33:57.227829 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.227811 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-clkt5\"/\"openshift-service-ca.crt\"" Jun 03 16:33:57.240043 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.240014 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-clkt5/must-gather-97xb6"] Jun 03 16:33:57.375198 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.375163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-272vq\" (UniqueName: \"kubernetes.io/projected/37ce0495-b1bd-4186-946b-b428136c9eee-kube-api-access-272vq\") pod \"must-gather-97xb6\" (UID: \"37ce0495-b1bd-4186-946b-b428136c9eee\") " pod="openshift-must-gather-clkt5/must-gather-97xb6" Jun 03 16:33:57.375377 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.375338 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/37ce0495-b1bd-4186-946b-b428136c9eee-must-gather-output\") pod \"must-gather-97xb6\" (UID: \"37ce0495-b1bd-4186-946b-b428136c9eee\") " pod="openshift-must-gather-clkt5/must-gather-97xb6" Jun 03 16:33:57.476353 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.476266 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-272vq\" (UniqueName: \"kubernetes.io/projected/37ce0495-b1bd-4186-946b-b428136c9eee-kube-api-access-272vq\") pod \"must-gather-97xb6\" (UID: \"37ce0495-b1bd-4186-946b-b428136c9eee\") " pod="openshift-must-gather-clkt5/must-gather-97xb6" Jun 03 16:33:57.476353 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.476344 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/37ce0495-b1bd-4186-946b-b428136c9eee-must-gather-output\") pod \"must-gather-97xb6\" (UID: \"37ce0495-b1bd-4186-946b-b428136c9eee\") " pod="openshift-must-gather-clkt5/must-gather-97xb6" Jun 03 16:33:57.476626 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.476612 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/37ce0495-b1bd-4186-946b-b428136c9eee-must-gather-output\") pod \"must-gather-97xb6\" (UID: \"37ce0495-b1bd-4186-946b-b428136c9eee\") " pod="openshift-must-gather-clkt5/must-gather-97xb6" Jun 03 16:33:57.486643 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.486610 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-272vq\" (UniqueName: \"kubernetes.io/projected/37ce0495-b1bd-4186-946b-b428136c9eee-kube-api-access-272vq\") pod \"must-gather-97xb6\" (UID: \"37ce0495-b1bd-4186-946b-b428136c9eee\") " pod="openshift-must-gather-clkt5/must-gather-97xb6" Jun 03 16:33:57.534563 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.534529 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-clkt5/must-gather-97xb6" Jun 03 16:33:57.667672 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:57.667632 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-clkt5/must-gather-97xb6"] Jun 03 16:33:57.670521 ip-10-0-139-61 kubenswrapper[2563]: W0603 16:33:57.670482 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37ce0495_b1bd_4186_946b_b428136c9eee.slice/crio-819eec8f1e2070795c9a0354b63399694df88c700d4d845e14007bf3c7a46b51 WatchSource:0}: Error finding container 819eec8f1e2070795c9a0354b63399694df88c700d4d845e14007bf3c7a46b51: Status 404 returned error can't find the container with id 819eec8f1e2070795c9a0354b63399694df88c700d4d845e14007bf3c7a46b51 Jun 03 16:33:58.504710 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:58.504673 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-clkt5/must-gather-97xb6" event={"ID":"37ce0495-b1bd-4186-946b-b428136c9eee","Type":"ContainerStarted","Data":"819eec8f1e2070795c9a0354b63399694df88c700d4d845e14007bf3c7a46b51"} Jun 03 16:33:59.512251 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:59.512211 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-clkt5/must-gather-97xb6" event={"ID":"37ce0495-b1bd-4186-946b-b428136c9eee","Type":"ContainerStarted","Data":"f940e8653be449f9071eba98e40fd790054f9bd2dc25f7a5f264c3a389a84a98"} Jun 03 16:33:59.512805 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:59.512781 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-clkt5/must-gather-97xb6" event={"ID":"37ce0495-b1bd-4186-946b-b428136c9eee","Type":"ContainerStarted","Data":"f53293ea1e56aca9080762b8ce4c0a0b648f79e249a8b3f592af715601cd0451"} Jun 03 16:33:59.530247 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:33:59.530188 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-clkt5/must-gather-97xb6" podStartSLOduration=1.651003497 podStartE2EDuration="2.530170913s" podCreationTimestamp="2026-06-03 16:33:57 +0000 UTC" firstStartedPulling="2026-06-03 16:33:57.672285041 +0000 UTC m=+1001.967478061" lastFinishedPulling="2026-06-03 16:33:58.551452458 +0000 UTC m=+1002.846645477" observedRunningTime="2026-06-03 16:33:59.527466249 +0000 UTC m=+1003.822659290" watchObservedRunningTime="2026-06-03 16:33:59.530170913 +0000 UTC m=+1003.825363955" Jun 03 16:34:00.235461 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:00.235427 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gx7b4_618bb3a0-8cda-4675-a219-c4b2aefb8f0f/global-pull-secret-syncer/0.log" Jun 03 16:34:00.388683 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:00.388608 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sww98_60c658c6-ac70-44dd-9644-69ed51bef4e1/konnectivity-agent/0.log" Jun 03 16:34:00.459832 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:00.459792 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-61.ec2.internal_34dbc6c38db1d2dc2bb23682a33284f5/haproxy/0.log" Jun 03 16:34:04.943842 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:04.943805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69596fbcfc-94zwp_3d5ef6f6-bc37-4da1-a1e8-d49c1eb9e04e/authorino/0.log" Jun 03 16:34:05.151096 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:05.151060 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-w8hh2_8a177361-44ee-4311-a1aa-adb6b54b26ad/manager/0.log" Jun 03 16:34:05.178207 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:05.178178 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-9tcgm_889de063-6ecd-4af4-9454-a87b9bea77dc/limitador/0.log" Jun 03 16:34:06.929539 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:06.929510 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-fxhbg_2ba6e461-7b60-47f7-befb-5fa6e2bb517b/kube-state-metrics/0.log" Jun 03 16:34:06.953973 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:06.953929 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-fxhbg_2ba6e461-7b60-47f7-befb-5fa6e2bb517b/kube-rbac-proxy-main/0.log" Jun 03 16:34:06.980074 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:06.980032 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-fxhbg_2ba6e461-7b60-47f7-befb-5fa6e2bb517b/kube-rbac-proxy-self/0.log" Jun 03 16:34:07.016649 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.016623 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c8d9f5b4d-wt5zl_7ac826a1-68da-4dfd-aa53-297d179cc1f4/metrics-server/0.log" Jun 03 16:34:07.272727 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.272635 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ltlkt_3ef15a1d-fef4-47aa-9035-72b5d1a991f0/node-exporter/0.log" Jun 03 16:34:07.300152 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.300121 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ltlkt_3ef15a1d-fef4-47aa-9035-72b5d1a991f0/kube-rbac-proxy/0.log" Jun 03 16:34:07.325126 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.325101 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ltlkt_3ef15a1d-fef4-47aa-9035-72b5d1a991f0/init-textfile/0.log" Jun 03 16:34:07.360832 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.360801 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-444lz_24a22ac6-b3b9-4472-ac95-f13671a712c1/kube-rbac-proxy-main/0.log" Jun 03 16:34:07.390384 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.390345 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-444lz_24a22ac6-b3b9-4472-ac95-f13671a712c1/kube-rbac-proxy-self/0.log" Jun 03 16:34:07.416130 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.416097 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-444lz_24a22ac6-b3b9-4472-ac95-f13671a712c1/openshift-state-metrics/0.log" Jun 03 16:34:07.464214 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.464183 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2abe109e-4130-41ce-b2c0-038c8cf4b1bc/prometheus/0.log" Jun 03 16:34:07.493011 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.492969 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2abe109e-4130-41ce-b2c0-038c8cf4b1bc/config-reloader/0.log" Jun 03 16:34:07.526070 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.525934 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2abe109e-4130-41ce-b2c0-038c8cf4b1bc/thanos-sidecar/0.log" Jun 03 16:34:07.558292 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.558259 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2abe109e-4130-41ce-b2c0-038c8cf4b1bc/kube-rbac-proxy-web/0.log" Jun 03 16:34:07.590526 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.590494 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2abe109e-4130-41ce-b2c0-038c8cf4b1bc/kube-rbac-proxy/0.log" Jun 03 16:34:07.618877 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.618851 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2abe109e-4130-41ce-b2c0-038c8cf4b1bc/kube-rbac-proxy-thanos/0.log" Jun 03 16:34:07.643983 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:07.643913 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2abe109e-4130-41ce-b2c0-038c8cf4b1bc/init-config-reloader/0.log" Jun 03 16:34:09.199539 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.199502 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5"] Jun 03 16:34:09.204645 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.204619 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.214365 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.214337 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5"] Jun 03 16:34:09.292900 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.292855 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-podres\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.293116 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.292910 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4c5m\" (UniqueName: \"kubernetes.io/projected/07d44d67-1800-4b94-951d-5174851b5081-kube-api-access-v4c5m\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.293116 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.292964 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-sys\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.293116 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.293015 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-lib-modules\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.293116 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.293048 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-proc\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.393761 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.393720 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-lib-modules\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.393919 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.393770 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-proc\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.393919 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.393857 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-podres\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.394035 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.394020 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v4c5m\" (UniqueName: \"kubernetes.io/projected/07d44d67-1800-4b94-951d-5174851b5081-kube-api-access-v4c5m\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.394072 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.394051 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-sys\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.394112 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.393906 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-lib-modules\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.394112 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.393934 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-proc\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.394196 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.393979 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-podres\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.394196 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.394138 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07d44d67-1800-4b94-951d-5174851b5081-sys\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.403294 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.403249 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4c5m\" (UniqueName: \"kubernetes.io/projected/07d44d67-1800-4b94-951d-5174851b5081-kube-api-access-v4c5m\") pod \"perf-node-gather-daemonset-kbdc5\" (UID: \"07d44d67-1800-4b94-951d-5174851b5081\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.518214 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.518143 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:09.701906 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:09.701730 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5"] Jun 03 16:34:10.380528 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:10.380497 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-shkcj_f534aa01-28bb-4e35-83e5-9494f96eb8de/download-server/0.log" Jun 03 16:34:10.564121 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:10.564082 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" event={"ID":"07d44d67-1800-4b94-951d-5174851b5081","Type":"ContainerStarted","Data":"bdb252a810099fea552610f495e7d108856321f683384886c24ee890ef7bd028"} Jun 03 16:34:10.564121 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:10.564119 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" event={"ID":"07d44d67-1800-4b94-951d-5174851b5081","Type":"ContainerStarted","Data":"ee2e81b58790203500af329ca8b6ad3ad9b29bb137262aed47f07fd94fae5932"} Jun 03 16:34:10.564366 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:10.564234 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:10.583347 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:10.583300 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" podStartSLOduration=1.583286089 podStartE2EDuration="1.583286089s" podCreationTimestamp="2026-06-03 16:34:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:34:10.581170056 +0000 UTC m=+1014.876363097" watchObservedRunningTime="2026-06-03 16:34:10.583286089 +0000 UTC m=+1014.878479130" Jun 03 16:34:11.800006 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:11.799975 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-czpwp_08e7d1e6-c8f1-421b-8210-138ca76e7354/dns/0.log" Jun 03 16:34:11.824202 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:11.824168 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-czpwp_08e7d1e6-c8f1-421b-8210-138ca76e7354/kube-rbac-proxy/0.log" Jun 03 16:34:11.928053 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:11.928024 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g279r_cfefc2c0-1f78-4bec-b7cd-9b3e7a6fc09a/dns-node-resolver/0.log" Jun 03 16:34:12.481334 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:12.481307 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-d4f7b9487-mb7cs_7c19a3f9-7c3a-4eb1-be3b-d6595206f370/registry/0.log" Jun 03 16:34:12.565437 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:12.565412 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mqcn6_175e42a6-fdf2-4684-91cb-53d33a7bc0fd/node-ca/0.log" Jun 03 16:34:13.591744 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:13.591716 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b4fb99578-2dwtm_91b33d16-f254-4940-8265-039e65e0059a/kube-auth-proxy/0.log" Jun 03 16:34:14.290803 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:14.290775 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7rsgf_deec8169-7f63-4f6d-bcfb-f35deee23591/serve-healthcheck-canary/0.log" Jun 03 16:34:14.977249 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:14.977220 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mn9rf_e4f8d59e-c5ad-4920-8316-f8d4451d1ec1/kube-rbac-proxy/0.log" Jun 03 16:34:15.000486 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:15.000458 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mn9rf_e4f8d59e-c5ad-4920-8316-f8d4451d1ec1/exporter/0.log" Jun 03 16:34:15.023685 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:15.023662 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mn9rf_e4f8d59e-c5ad-4920-8316-f8d4451d1ec1/extractor/0.log" Jun 03 16:34:16.578763 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:16.578735 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-kbdc5" Jun 03 16:34:16.995317 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:16.995292 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29675070-5tx49_9699b5cd-a58a-471a-86f5-8eee00d4e0a9/cleanup/0.log" Jun 03 16:34:17.166899 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:17.166874 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c656db94d-k6nfz_16dc22cf-d4e4-41e8-9beb-7b2f3217a46f/manager/0.log" Jun 03 16:34:18.737806 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:18.737775 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bd65fcf99-4vfjk_5ce1b328-c5bd-4a3c-97ed-9e6356a55b6b/manager/0.log" Jun 03 16:34:24.694273 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:24.694237 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9q2rw_e16c3c6f-c435-43c3-a3cc-cd43071c7412/kube-multus/0.log" Jun 03 16:34:24.881111 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:24.881084 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlqp5_f754daa3-4c89-4f39-a9d3-91b61ce13bca/kube-multus-additional-cni-plugins/0.log" Jun 03 16:34:24.904425 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:24.904399 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlqp5_f754daa3-4c89-4f39-a9d3-91b61ce13bca/egress-router-binary-copy/0.log" Jun 03 16:34:24.927134 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:24.927099 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlqp5_f754daa3-4c89-4f39-a9d3-91b61ce13bca/cni-plugins/0.log" Jun 03 16:34:24.949384 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:24.949321 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlqp5_f754daa3-4c89-4f39-a9d3-91b61ce13bca/bond-cni-plugin/0.log" Jun 03 16:34:24.972550 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:24.972524 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlqp5_f754daa3-4c89-4f39-a9d3-91b61ce13bca/routeoverride-cni/0.log" Jun 03 16:34:24.998146 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:24.998120 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlqp5_f754daa3-4c89-4f39-a9d3-91b61ce13bca/whereabouts-cni-bincopy/0.log" Jun 03 16:34:25.027258 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:25.027226 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlqp5_f754daa3-4c89-4f39-a9d3-91b61ce13bca/whereabouts-cni/0.log" Jun 03 16:34:25.234757 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:25.234691 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2xsw7_0ac3b8bc-dd32-4bf5-97eb-84edbc66f369/network-metrics-daemon/0.log" Jun 03 16:34:25.258562 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:25.258538 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2xsw7_0ac3b8bc-dd32-4bf5-97eb-84edbc66f369/kube-rbac-proxy/0.log" Jun 03 16:34:26.214317 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.214283 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-controller/0.log" Jun 03 16:34:26.235148 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.235122 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/0.log" Jun 03 16:34:26.240168 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.240145 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovn-acl-logging/1.log" Jun 03 16:34:26.258393 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.258370 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/kube-rbac-proxy-node/0.log" Jun 03 16:34:26.281445 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.281369 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/kube-rbac-proxy-ovn-metrics/0.log" Jun 03 16:34:26.303633 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.303604 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/northd/0.log" Jun 03 16:34:26.326810 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.326785 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/nbdb/0.log" Jun 03 16:34:26.350200 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.350165 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/sbdb/0.log" Jun 03 16:34:26.477320 ip-10-0-139-61 kubenswrapper[2563]: I0603 16:34:26.477286 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htflp_51336783-9c76-4705-8d2a-d9f3ad0e6089/ovnkube-controller/0.log"