Jun 08 12:04:26.291849 ip-10-0-142-230 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 08 12:04:26.291862 ip-10-0-142-230 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 08 12:04:26.291872 ip-10-0-142-230 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 08 12:04:26.292070 ip-10-0-142-230 systemd[1]: Failed to start Kubernetes Kubelet. Jun 08 12:04:36.402976 ip-10-0-142-230 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 08 12:04:36.402995 ip-10-0-142-230 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d7b85dfc13e54fee96d34d7cdf3c3359 -- Jun 08 12:07:17.208300 ip-10-0-142-230 systemd[1]: Starting Kubernetes Kubelet... Jun 08 12:07:17.697231 ip-10-0-142-230 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 12:07:17.697231 ip-10-0-142-230 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 08 12:07:17.697231 ip-10-0-142-230 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 12:07:17.697231 ip-10-0-142-230 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 08 12:07:17.697231 ip-10-0-142-230 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 12:07:17.699098 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.698954 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 08 12:07:17.705381 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705361 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:17.705381 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705377 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705387 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705390 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705393 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705396 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705401 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705404 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705406 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705410 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705413 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705416 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705418 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705420 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705423 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705426 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705429 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705431 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705434 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705436 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705439 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:17.705451 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705442 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705444 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705449 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705452 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705456 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705459 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705462 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705468 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705471 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705475 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705477 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705480 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705483 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705486 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705489 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705492 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705495 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705498 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705501 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:17.705928 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705503 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705506 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705509 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705511 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705514 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705517 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705520 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705522 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705526 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705530 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705532 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705534 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705537 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705540 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705542 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705546 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705550 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705552 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705555 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:17.706454 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705558 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705561 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705563 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705565 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705568 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705571 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705574 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705576 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705579 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705582 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705584 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705587 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705589 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705592 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705596 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705598 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705600 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705603 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705606 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:17.707186 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705608 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705610 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705613 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705616 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705618 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705621 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705623 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705627 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705630 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705632 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705636 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705638 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705641 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705644 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705646 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705649 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705651 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705654 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705657 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705659 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:17.707711 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.705661 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707654 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707668 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707672 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707675 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707678 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707681 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707683 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707686 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707689 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707691 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707694 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707697 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707699 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707702 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707704 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707707 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707709 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707712 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707715 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:17.708216 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707718 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707721 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707725 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707727 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707730 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707732 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707735 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707738 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707740 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707743 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707746 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707748 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707752 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707757 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707761 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707764 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707766 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707769 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707772 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:17.708736 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707774 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707777 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707779 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707782 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707785 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707787 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707790 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707792 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707795 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707798 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707801 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707804 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707808 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707810 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707813 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707815 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707818 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707821 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707823 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707826 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:17.709224 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707828 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707831 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707835 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707840 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707842 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707845 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707848 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707851 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707854 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707857 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707859 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707862 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707864 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707867 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707869 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707872 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707874 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707877 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707879 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:17.710283 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707882 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707887 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707890 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707893 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707903 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707906 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707918 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707921 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707923 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707926 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707929 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707931 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707934 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707937 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707951 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707954 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707957 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707960 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707962 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707965 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:17.710754 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707971 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.707973 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708354 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708359 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708362 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708365 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708368 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708371 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708374 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708376 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708379 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708382 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708384 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708387 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708389 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708392 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708395 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708398 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708402 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708404 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:17.711273 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708407 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708409 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708412 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708414 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708417 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708419 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708422 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708424 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708426 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708429 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708432 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708434 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708437 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708440 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708442 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708445 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708447 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708450 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708452 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708455 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:17.711753 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708458 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708460 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708463 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708465 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708468 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708471 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708473 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708475 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708478 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708481 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708483 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708487 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708490 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708493 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708495 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708498 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708501 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708503 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708506 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708508 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:17.712319 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708511 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708513 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708516 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708519 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708521 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708524 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708527 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708529 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708532 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708535 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708537 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708540 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708542 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708544 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708547 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708549 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708552 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708555 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708557 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:17.712815 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708560 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708565 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708570 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708574 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708577 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708581 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708584 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708587 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708590 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708592 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708595 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708597 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708600 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708603 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708605 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708607 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708610 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708612 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:17.713317 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708615 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708619 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708622 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.708624 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708695 2580 flags.go:64] FLAG: --address="0.0.0.0" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708702 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708708 2580 flags.go:64] FLAG: --anonymous-auth="true" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708713 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708717 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708720 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708724 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708728 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708731 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708734 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708738 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708741 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708744 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708748 2580 flags.go:64] FLAG: --cgroup-root="" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708751 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708754 2580 flags.go:64] FLAG: --client-ca-file="" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708758 2580 flags.go:64] FLAG: --cloud-provider="external" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708761 2580 flags.go:64] FLAG: --cluster-dns="[]" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708765 2580 flags.go:64] FLAG: --cluster-domain="" Jun 08 12:07:17.713757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708768 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708771 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708774 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708778 2580 flags.go:64] FLAG: --container-log-max-files="5" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708781 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708784 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708787 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708790 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708793 2580 flags.go:64] FLAG: --contention-profiling="false" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708796 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708799 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708802 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708805 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708809 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708812 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708814 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708817 2580 flags.go:64] FLAG: --enable-load-reader="false" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708820 2580 flags.go:64] FLAG: --enable-server="true" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708823 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708828 2580 flags.go:64] FLAG: --event-burst="100" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708831 2580 flags.go:64] FLAG: --event-qps="50" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708833 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708836 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708839 2580 flags.go:64] FLAG: --eviction-hard="" Jun 08 12:07:17.714329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708843 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708849 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708852 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708855 2580 flags.go:64] FLAG: --eviction-soft="" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708858 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708863 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708870 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708873 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708876 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708879 2580 flags.go:64] FLAG: --fail-swap-on="true" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708882 2580 flags.go:64] FLAG: --feature-gates="" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708886 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708889 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708891 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708894 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708898 2580 flags.go:64] FLAG: --healthz-port="10248" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708901 2580 flags.go:64] FLAG: --help="false" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708904 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-142-230.ec2.internal" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708907 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708909 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708912 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708916 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708919 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708922 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 08 12:07:17.714926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708925 2580 flags.go:64] FLAG: --image-service-endpoint="" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708928 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708931 2580 flags.go:64] FLAG: --kube-api-burst="100" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708934 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708937 2580 flags.go:64] FLAG: --kube-api-qps="50" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708955 2580 flags.go:64] FLAG: --kube-reserved="" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708958 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708961 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708964 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708968 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708971 2580 flags.go:64] FLAG: --lock-file="" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708974 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708977 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708981 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708987 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708990 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708993 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708996 2580 flags.go:64] FLAG: --logging-format="text" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.708999 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709002 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709005 2580 flags.go:64] FLAG: --manifest-url="" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709008 2580 flags.go:64] FLAG: --manifest-url-header="" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709012 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709015 2580 flags.go:64] FLAG: --max-open-files="1000000" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709019 2580 flags.go:64] FLAG: --max-pods="110" Jun 08 12:07:17.715612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709022 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709025 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709028 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709031 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709034 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709037 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709040 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709047 2580 flags.go:64] FLAG: --node-status-max-images="50" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709050 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709053 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709056 2580 flags.go:64] FLAG: --pod-cidr="" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709059 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709064 2580 flags.go:64] FLAG: --pod-manifest-path="" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709067 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709070 2580 flags.go:64] FLAG: --pods-per-core="0" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709073 2580 flags.go:64] FLAG: --port="10250" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709076 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709081 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d1ad3ad976158d70" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709084 2580 flags.go:64] FLAG: --qos-reserved="" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709087 2580 flags.go:64] FLAG: --read-only-port="10255" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709091 2580 flags.go:64] FLAG: --register-node="true" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709094 2580 flags.go:64] FLAG: --register-with-taints="" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709098 2580 flags.go:64] FLAG: --registry-burst="10" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709102 2580 flags.go:64] FLAG: --registry-qps="5" Jun 08 12:07:17.716247 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709105 2580 flags.go:64] FLAG: --reserved-cpus="" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709107 2580 flags.go:64] FLAG: --reserved-memory="" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709111 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709114 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709117 2580 flags.go:64] FLAG: --rotate-certificates="false" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709120 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709123 2580 flags.go:64] FLAG: --runonce="false" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709126 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709129 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709132 2580 flags.go:64] FLAG: --seccomp-default="false" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709135 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709138 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709141 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709144 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709147 2580 flags.go:64] FLAG: --storage-driver-password="root" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709149 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709152 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709155 2580 flags.go:64] FLAG: --storage-driver-user="root" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709158 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709160 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709163 2580 flags.go:64] FLAG: --system-cgroups="" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709166 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709172 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709174 2580 flags.go:64] FLAG: --tls-cert-file="" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709177 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 08 12:07:17.716831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709182 2580 flags.go:64] FLAG: --tls-min-version="" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709185 2580 flags.go:64] FLAG: --tls-private-key-file="" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709188 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709192 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709195 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709197 2580 flags.go:64] FLAG: --v="2" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709202 2580 flags.go:64] FLAG: --version="false" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709206 2580 flags.go:64] FLAG: --vmodule="" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709210 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709213 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709327 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709331 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709335 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709338 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709341 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709344 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709347 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709351 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709355 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709358 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709361 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:17.717511 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709363 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709366 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709369 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709372 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709374 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709377 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709380 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709382 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709385 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709388 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709390 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709395 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709398 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709400 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709404 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709407 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709409 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709412 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709415 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709418 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:17.718046 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709420 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709423 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709425 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709428 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709431 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709433 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709436 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709439 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709441 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709444 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709446 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709449 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709451 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709454 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709457 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709459 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709462 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709464 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709469 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709472 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:17.718520 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709475 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709478 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709481 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709485 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709488 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709491 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709496 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709499 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709501 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709504 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709506 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709509 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709516 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709519 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709521 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709524 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709526 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709529 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709532 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709535 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:17.719091 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709537 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709540 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709542 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709545 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709548 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709550 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709553 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709555 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709558 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709561 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709563 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709566 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709568 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709571 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709573 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709578 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709580 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709583 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709587 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709590 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:17.719585 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709593 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709596 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709599 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709602 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709606 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709614 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709617 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.709621 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.709626 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.716422 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.716442 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.717449 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.717457 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717521 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:17.720107 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717527 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717531 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717535 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717540 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717544 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717548 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717550 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717553 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717556 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717559 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717562 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717564 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717567 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717570 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717572 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717575 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717577 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717580 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717583 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717585 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:17.720463 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717604 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717608 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717611 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717614 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717618 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717620 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717624 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717626 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717631 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717634 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717638 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717641 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717644 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717646 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717650 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717653 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717656 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717658 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717661 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:17.720983 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717664 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717666 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717669 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717672 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717675 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717679 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717682 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717685 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717688 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717690 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717693 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717695 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717698 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717701 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717703 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717706 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717709 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717711 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717714 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:17.721447 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717717 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717720 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717725 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717728 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717731 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717734 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717736 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717739 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717742 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717745 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717747 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717750 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717752 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717755 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717757 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717760 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717762 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717765 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717767 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:17.721900 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717770 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717772 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717775 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717777 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717780 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717783 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717785 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717788 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717791 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717794 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717796 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717799 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717801 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717804 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717807 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717810 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717812 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717815 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717818 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717821 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:17.722431 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717823 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.717828 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717976 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717981 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717984 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717987 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717990 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717992 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717995 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.717998 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718001 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718003 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718006 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718009 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718011 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:17.722967 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718014 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718017 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718019 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718022 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718025 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718028 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718030 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718033 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718036 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718038 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718041 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718043 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718046 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718049 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718051 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718055 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718058 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718061 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718063 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718066 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:17.723354 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718068 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718071 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718073 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718076 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718079 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718083 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718085 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718088 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718091 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718093 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718096 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718098 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718102 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718106 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718109 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718112 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718115 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718117 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718120 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:17.723828 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718122 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718125 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718128 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718130 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718133 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718135 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718138 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718140 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718143 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718146 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718148 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718151 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718153 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718156 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718158 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718161 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718163 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718165 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718168 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718170 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:17.724300 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718173 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718175 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718177 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718180 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718183 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718186 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718188 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718191 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718193 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718196 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718199 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718203 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718206 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718208 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718211 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718214 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718217 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718219 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718222 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:17.724775 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718224 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718227 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718230 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718233 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718235 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718238 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718240 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:17.718243 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.718248 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 12:07:17.725257 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.718353 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jun 08 12:07:17.725593 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.725578 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 08 12:07:17.726982 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.726970 2580 server.go:1019] "Starting client certificate rotation" Jun 08 12:07:17.727107 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.727091 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 12:07:17.727142 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.727122 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 12:07:17.729861 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.729849 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 12:07:17.731358 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.731335 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 12:07:17.744723 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.744703 2580 log.go:25] "Validated CRI v1 runtime API" Jun 08 12:07:17.747501 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.747487 2580 log.go:25] "Validated CRI v1 image API" Jun 08 12:07:17.748351 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.748334 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 08 12:07:17.753405 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.753384 2580 fs.go:135] Filesystem UUIDs: map[084add8c-0586-4f59-8867-48fdf270ef3e:/dev/nvme0n1p4 5d6ad429-88b7-4a8f-bda3-84ae98f59aa4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jun 08 12:07:17.753501 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.753404 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 08 12:07:17.753714 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.753696 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 12:07:17.757718 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.757601 2580 manager.go:216] Machine: {Timestamp:2026-06-08 12:07:17.757134049 +0000 UTC m=+0.396975474 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096584 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b390e786f3c536ed263c48669de6d SystemUUID:ec2b390e-786f-3c53-6ed2-63c48669de6d BootID:d7b85dfc-13e5-4fee-96d3-4d7cdf3c3359 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bd:16:0e:a1:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bd:16:0e:a1:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:66:ec:63:f7:d4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 08 12:07:17.757718 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.757709 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 08 12:07:17.757871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.757812 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 08 12:07:17.761538 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.761510 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 08 12:07:17.761707 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.761540 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 08 12:07:17.762571 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.762560 2580 topology_manager.go:138] "Creating topology manager with none policy" Jun 08 12:07:17.762636 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.762580 2580 container_manager_linux.go:309] "Creating device plugin manager" Jun 08 12:07:17.762636 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.762591 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 12:07:17.763391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.763379 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 12:07:17.764242 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.764231 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 08 12:07:17.767009 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.766995 2580 state_mem.go:36] "Initialized new in-memory state store" Jun 08 12:07:17.767147 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.767135 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 08 12:07:17.767202 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.767173 2580 kubelet.go:486] "Attempting to sync node with API server" Jun 08 12:07:17.767202 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.767188 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 08 12:07:17.767295 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.767208 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 08 12:07:17.767295 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.767221 2580 kubelet.go:398] "Adding apiserver pod source" Jun 08 12:07:17.767295 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.767243 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 08 12:07:17.767632 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.767617 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 12:07:17.767699 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.767641 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 12:07:17.770429 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.770413 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 08 12:07:17.771894 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.771881 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 08 12:07:17.771960 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.771911 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 08 12:07:17.773065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773053 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773069 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773076 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773081 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773090 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773095 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773103 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773108 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773116 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 08 12:07:17.773128 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773122 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 08 12:07:17.773379 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773135 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 08 12:07:17.773379 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773141 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 08 12:07:17.773379 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773166 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 08 12:07:17.773379 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.773171 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 08 12:07:17.775432 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.775416 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f69j8" Jun 08 12:07:17.776621 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.776607 2580 server.go:1268] "Started kubelet" Jun 08 12:07:17.777406 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.776800 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 08 12:07:17.777406 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.777096 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 08 12:07:17.777406 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.777330 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 08 12:07:17.777535 ip-10-0-142-230 systemd[1]: Started Kubernetes Kubelet. Jun 08 12:07:17.778013 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.777996 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 08 12:07:17.778317 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.778285 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 08 12:07:17.778378 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.778309 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 08 12:07:17.779338 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.779199 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 08 12:07:17.779532 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.779519 2580 server.go:310] "Adding debug handlers to kubelet server" Jun 08 12:07:17.783144 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.783124 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f69j8" Jun 08 12:07:17.784809 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.783791 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-230.ec2.internal.18b719b3d2e7506a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-230.ec2.internal,UID:ip-10-0-142-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-230.ec2.internal,},FirstTimestamp:2026-06-08 12:07:17.776535658 +0000 UTC m=+0.416377085,LastTimestamp:2026-06-08 12:07:17.776535658 +0000 UTC m=+0.416377085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-230.ec2.internal,}" Jun 08 12:07:17.785647 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.785629 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 08 12:07:17.786031 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.786012 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 08 12:07:17.786031 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.786029 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 08 12:07:17.786185 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.786167 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:17.786771 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.786754 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 08 12:07:17.786872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.786762 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 08 12:07:17.786990 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.786886 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 08 12:07:17.786990 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.786965 2580 reconstruct.go:97] "Volume reconstruction finished" Jun 08 12:07:17.786990 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.786974 2580 reconciler.go:29] "Reconciler: start to sync state" Jun 08 12:07:17.787113 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.787100 2580 factory.go:55] Registering systemd factory Jun 08 12:07:17.787151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.787118 2580 factory.go:223] Registration of the systemd container factory successfully Jun 08 12:07:17.787332 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.787318 2580 factory.go:153] Registering CRI-O factory Jun 08 12:07:17.787405 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.787336 2580 factory.go:223] Registration of the crio container factory successfully Jun 08 12:07:17.787405 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.787385 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 08 12:07:17.787495 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.787408 2580 factory.go:103] Registering Raw factory Jun 08 12:07:17.787495 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.787423 2580 manager.go:1196] Started watching for new ooms in manager Jun 08 12:07:17.788109 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.788092 2580 manager.go:349] Starting recovery of all containers Jun 08 12:07:17.789639 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.789608 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 08 12:07:17.789720 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.789696 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 08 12:07:17.799593 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.799444 2580 manager.go:354] Recovery completed Jun 08 12:07:17.803898 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.803883 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:17.805870 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.805855 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:17.805929 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.805882 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:17.805929 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.805893 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:17.806843 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.806829 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 08 12:07:17.806843 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.806841 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 08 12:07:17.806994 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.806855 2580 state_mem.go:36] "Initialized new in-memory state store" Jun 08 12:07:17.806994 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.806882 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:17.807686 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.807669 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:17.807749 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.807698 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:17.807749 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.807710 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:17.808084 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.808069 2580 policy_none.go:49] "None policy: Start" Jun 08 12:07:17.808116 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.808091 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 08 12:07:17.808116 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.808105 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 08 12:07:17.809114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.809034 2580 policy_none.go:47] "Start" Jun 08 12:07:17.844497 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.844477 2580 manager.go:337] "Starting Device Plugin manager" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.844526 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.844539 2580 server.go:87] "Starting device plugin registration server" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.844767 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.844779 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.844975 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.845483 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.845494 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.848009 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.848061 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:17.853561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.852881 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 08 12:07:17.854048 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.854033 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 08 12:07:17.854088 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.854073 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 08 12:07:17.854122 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.854093 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jun 08 12:07:17.854327 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.854128 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 08 12:07:17.856369 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.856350 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:17.945385 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.945366 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:17.946925 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.946908 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:17.947040 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.946959 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:17.947040 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.946975 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:17.947040 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.947006 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-230.ec2.internal" Jun 08 12:07:17.952636 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.952596 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-230.ec2.internal" Jun 08 12:07:17.952636 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.952618 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-230.ec2.internal\": node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:17.954996 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.954981 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"] Jun 08 12:07:17.955046 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.955037 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:17.955860 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.955842 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:17.955957 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.955873 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:17.955957 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.955886 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:17.957069 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957054 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:17.957162 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957147 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:17.957206 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957178 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:17.957695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957681 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:17.957786 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957701 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:17.957786 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957712 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:17.957786 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957718 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:17.957786 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957740 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:17.957786 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.957753 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:17.958925 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.958908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jun 08 12:07:17.959020 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.958954 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:17.959542 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.959528 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:17.959638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.959562 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:17.959638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:17.959576 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:17.976895 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.976875 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:17.987526 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.987505 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-230.ec2.internal\" not found" node="ip-10-0-142-230.ec2.internal" Jun 08 12:07:17.991862 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:17.991845 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-230.ec2.internal\" not found" node="ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.077623 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.077598 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.087526 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.087506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/397bf72bf63ce09189ff4e95e7b44b6a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"397bf72bf63ce09189ff4e95e7b44b6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.087586 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.087532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/397bf72bf63ce09189ff4e95e7b44b6a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"397bf72bf63ce09189ff4e95e7b44b6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.087586 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.087551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba29b9ee45638b00406cbe12817dc6fb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"ba29b9ee45638b00406cbe12817dc6fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.177947 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.177925 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.188213 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.188191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/397bf72bf63ce09189ff4e95e7b44b6a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"397bf72bf63ce09189ff4e95e7b44b6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.188266 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.188221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/397bf72bf63ce09189ff4e95e7b44b6a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"397bf72bf63ce09189ff4e95e7b44b6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.188266 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.188243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba29b9ee45638b00406cbe12817dc6fb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"ba29b9ee45638b00406cbe12817dc6fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.188329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.188281 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/397bf72bf63ce09189ff4e95e7b44b6a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"397bf72bf63ce09189ff4e95e7b44b6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.188329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.188300 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba29b9ee45638b00406cbe12817dc6fb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"ba29b9ee45638b00406cbe12817dc6fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.188329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.188281 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/397bf72bf63ce09189ff4e95e7b44b6a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"397bf72bf63ce09189ff4e95e7b44b6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.278548 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.278523 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.292951 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.292922 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.296238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.296216 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jun 08 12:07:18.378958 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.378917 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.479387 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.479355 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.579933 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.579863 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.654636 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.654611 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:18.680543 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.680514 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.726021 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.725994 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 08 12:07:18.726623 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.726143 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 12:07:18.726623 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.726147 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 12:07:18.781153 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.781130 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.788402 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.788352 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-07 12:02:17 +0000 UTC" deadline="2027-12-16 10:17:41.970680982 +0000 UTC" Jun 08 12:07:18.788515 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.788402 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13342h10m23.182283728s" Jun 08 12:07:18.788577 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:18.788508 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397bf72bf63ce09189ff4e95e7b44b6a.slice/crio-9f5c646870298b2e27ffdc7ec73911a1a18a64c3219224d9815e911a1286bc61 WatchSource:0}: Error finding container 9f5c646870298b2e27ffdc7ec73911a1a18a64c3219224d9815e911a1286bc61: Status 404 returned error can't find the container with id 9f5c646870298b2e27ffdc7ec73911a1a18a64c3219224d9815e911a1286bc61 Jun 08 12:07:18.788796 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:18.788780 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba29b9ee45638b00406cbe12817dc6fb.slice/crio-47e8da48fc9f1cdc52a162263e37eb7614f59f1d900e3379f650c93431180a84 WatchSource:0}: Error finding container 47e8da48fc9f1cdc52a162263e37eb7614f59f1d900e3379f650c93431180a84: Status 404 returned error can't find the container with id 47e8da48fc9f1cdc52a162263e37eb7614f59f1d900e3379f650c93431180a84 Jun 08 12:07:18.794878 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.794848 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 12:07:18.857872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.857773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" event={"ID":"ba29b9ee45638b00406cbe12817dc6fb","Type":"ContainerStarted","Data":"47e8da48fc9f1cdc52a162263e37eb7614f59f1d900e3379f650c93431180a84"} Jun 08 12:07:18.858748 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.858726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"397bf72bf63ce09189ff4e95e7b44b6a","Type":"ContainerStarted","Data":"9f5c646870298b2e27ffdc7ec73911a1a18a64c3219224d9815e911a1286bc61"} Jun 08 12:07:18.881903 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.881877 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:18.893073 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:18.893052 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:18.982403 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:18.982373 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:19.082925 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.082892 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jun 08 12:07:19.156813 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.156737 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:19.186303 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.186274 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jun 08 12:07:19.197951 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.197907 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 12:07:19.199277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.199254 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jun 08 12:07:19.207164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.207144 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 12:07:19.572720 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.572688 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:19.769131 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.769105 2580 apiserver.go:52] "Watching apiserver" Jun 08 12:07:19.777382 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.777358 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 08 12:07:19.777729 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.777700 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal","openshift-multus/multus-d2jfx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx","openshift-image-registry/node-ca-fhmb9","openshift-multus/multus-additional-cni-plugins-9n5nt","openshift-multus/network-metrics-daemon-wr6xl","openshift-network-diagnostics/network-check-target-9j7b8","openshift-network-operator/iptables-alerter-js9c4","openshift-ovn-kubernetes/ovnkube-node-j9m6c","kube-system/global-pull-secret-syncer-446pr","kube-system/konnectivity-agent-bshpx","kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wmwnv","openshift-dns/node-resolver-w469r"] Jun 08 12:07:19.779148 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.779127 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.781679 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.781651 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:19.782374 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.782342 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 08 12:07:19.782479 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.782399 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 08 12:07:19.782479 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.782349 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 08 12:07:19.782479 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.782442 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s2j2n\"" Jun 08 12:07:19.782640 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.782404 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 08 12:07:19.782720 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.782705 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 08 12:07:19.782789 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.782738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:19.782880 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.782858 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:19.783888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.783867 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:19.784001 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.783954 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:19.784071 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.784054 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 08 12:07:19.784450 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.784434 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dc5q9\"" Jun 08 12:07:19.784594 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.784568 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 08 12:07:19.785244 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.785223 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.786556 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.786504 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.786841 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.786806 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 08 12:07:19.787668 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.787650 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 08 12:07:19.787758 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.787672 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.787758 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.787730 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 08 12:07:19.787863 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.787782 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bgs7g\"" Jun 08 12:07:19.787919 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.787859 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 08 12:07:19.788755 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.788735 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 08 12:07:19.788879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.788808 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 08 12:07:19.788879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.788869 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6h5kb\"" Jun 08 12:07:19.789015 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.788989 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 08 12:07:19.789298 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.789280 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.790277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.790214 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 08 12:07:19.790277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.790271 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s9pcg\"" Jun 08 12:07:19.790411 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.790273 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 08 12:07:19.790499 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.790483 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 08 12:07:19.791119 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.791101 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:19.791204 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.791179 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:19.791926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.791906 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 08 12:07:19.792020 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.791964 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 08 12:07:19.792020 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.792011 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 08 12:07:19.792142 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.792023 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6s9rm\"" Jun 08 12:07:19.792581 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.792440 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.793205 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.793182 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 08 12:07:19.793295 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.793211 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 08 12:07:19.793353 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.793321 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 08 12:07:19.793854 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.793833 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.795126 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.795035 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 08 12:07:19.795126 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.795092 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4hrgl\"" Jun 08 12:07:19.795375 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.795357 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:19.796134 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.796117 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 08 12:07:19.796226 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.796140 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jxjtx\"" Jun 08 12:07:19.796226 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.796214 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 08 12:07:19.796885 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.796852 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kjck\" (UniqueName: \"kubernetes.io/projected/bfd9e9d8-9609-43d5-a45c-e58816b43118-kube-api-access-9kjck\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:19.797005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.796895 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg2s8\" (UniqueName: \"kubernetes.io/projected/66348e6c-fe0c-43c0-989c-fbd2c69373a1-kube-api-access-pg2s8\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.797005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.796918 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-sys-fs\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.797005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.796956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbntw\" (UniqueName: \"kubernetes.io/projected/d03715cd-70aa-4174-9885-64293af4ef3f-kube-api-access-fbntw\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.797005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.796986 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5991d28-df9c-45fe-a16b-dc6a4d1a25bd-konnectivity-ca\") pod \"konnectivity-agent-bshpx\" (UID: \"b5991d28-df9c-45fe-a16b-dc6a4d1a25bd\") " pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:19.797208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-registration-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.797208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.797208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.797208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg4bl\" (UniqueName: \"kubernetes.io/projected/48059286-7d75-419d-b43e-f7a83ee4fdb8-kube-api-access-kg4bl\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.797208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797105 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:19.797208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.797208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-socket-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.797208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:19.797605 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66348e6c-fe0c-43c0-989c-fbd2c69373a1-host\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.797605 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.797605 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797456 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 12:07:19.797908 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-cnibin\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.798048 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797930 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.798048 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-device-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.798048 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.797993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9581e35-486d-4e40-b4e7-96507433032c-host-slash\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.798048 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-system-cni-dir\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.798254 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-os-release\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.798254 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f9581e35-486d-4e40-b4e7-96507433032c-iptables-alerter-script\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.798254 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798165 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbz4l\" (UniqueName: \"kubernetes.io/projected/f9581e35-486d-4e40-b4e7-96507433032c-kube-api-access-lbz4l\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.798254 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.798254 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798197 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 08 12:07:19.798254 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798197 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 08 12:07:19.798538 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5991d28-df9c-45fe-a16b-dc6a4d1a25bd-agent-certs\") pod \"konnectivity-agent-bshpx\" (UID: \"b5991d28-df9c-45fe-a16b-dc6a4d1a25bd\") " pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:19.798538 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798270 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9cshs\"" Jun 08 12:07:19.798538 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.798298 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/66348e6c-fe0c-43c0-989c-fbd2c69373a1-serviceca\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.818416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.818321 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rtn8q" Jun 08 12:07:19.826569 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.826549 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rtn8q" Jun 08 12:07:19.888452 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.888427 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 08 12:07:19.898764 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9581e35-486d-4e40-b4e7-96507433032c-host-slash\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.898901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898768 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-os-release\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.898901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898795 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9581e35-486d-4e40-b4e7-96507433032c-host-slash\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.898901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898799 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-kubelet-config\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:19.898901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-dbus\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:19.898901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898874 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-os-release\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.898901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-cni-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898918 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-cni-bin\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-kubelet\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.898992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovn-node-metrics-cert\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwwbt\" (UniqueName: \"kubernetes.io/projected/0532dcdb-1611-488c-b8d3-6d84e84c687a-kube-api-access-qwwbt\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899030 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysctl-conf\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899057 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kjck\" (UniqueName: \"kubernetes.io/projected/bfd9e9d8-9609-43d5-a45c-e58816b43118-kube-api-access-9kjck\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbntw\" (UniqueName: \"kubernetes.io/projected/d03715cd-70aa-4174-9885-64293af4ef3f-kube-api-access-fbntw\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-hosts-file\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5991d28-df9c-45fe-a16b-dc6a4d1a25bd-konnectivity-ca\") pod \"konnectivity-agent-bshpx\" (UID: \"b5991d28-df9c-45fe-a16b-dc6a4d1a25bd\") " pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-system-cni-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899177 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-conf-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.899229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-etc-kubernetes\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-log-socket\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-cni-bin\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysconfig\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899370 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg4bl\" (UniqueName: \"kubernetes.io/projected/48059286-7d75-419d-b43e-f7a83ee4fdb8-kube-api-access-kg4bl\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-tuned\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-os-release\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-etc-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899622 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-cni-netd\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovnkube-script-lib\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-host\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-hostroot\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899765 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5991d28-df9c-45fe-a16b-dc6a4d1a25bd-konnectivity-ca\") pod \"konnectivity-agent-bshpx\" (UID: \"b5991d28-df9c-45fe-a16b-dc6a4d1a25bd\") " pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:19.899831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.899838 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-cnibin\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.899904 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:20.399882526 +0000 UTC m=+3.039723952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899905 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899914 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899936 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-cnibin\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899961 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-netns\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.899989 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-multus-certs\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-slash\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-run\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-system-cni-dir\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900149 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-system-cni-dir\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900190 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-kubelet\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900223 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-node-log\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.900669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysctl-d\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900281 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f9581e35-486d-4e40-b4e7-96507433032c-iptables-alerter-script\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900297 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbz4l\" (UniqueName: \"kubernetes.io/projected/f9581e35-486d-4e40-b4e7-96507433032c-kube-api-access-lbz4l\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5991d28-df9c-45fe-a16b-dc6a4d1a25bd-agent-certs\") pod \"konnectivity-agent-bshpx\" (UID: \"b5991d28-df9c-45fe-a16b-dc6a4d1a25bd\") " pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/66348e6c-fe0c-43c0-989c-fbd2c69373a1-serviceca\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900370 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-daemon-config\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-kubernetes\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-systemd\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg2s8\" (UniqueName: \"kubernetes.io/projected/66348e6c-fe0c-43c0-989c-fbd2c69373a1-kube-api-access-pg2s8\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-sys-fs\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovnkube-config\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-lib-modules\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-var-lib-kubelet\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-tmp\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900584 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-tmp-dir\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900657 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-sys-fs\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.901415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900713 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48059286-7d75-419d-b43e-f7a83ee4fdb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-registration-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900770 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w95ct\" (UniqueName: \"kubernetes.io/projected/419a276e-db0b-4bbb-acbb-31859a30e77c-kube-api-access-w95ct\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900812 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f9581e35-486d-4e40-b4e7-96507433032c-iptables-alerter-script\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-systemd-units\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900844 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-registration-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900851 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-run-netns\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-systemd\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900931 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-socket-dir-parent\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.900985 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-k8s-cni-cncf-io\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901042 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-cni-multus\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901063 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-sys\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5m4r\" (UniqueName: \"kubernetes.io/projected/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-kube-api-access-p5m4r\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901093 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flqpt\" (UniqueName: \"kubernetes.io/projected/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-kube-api-access-flqpt\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901041 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/66348e6c-fe0c-43c0-989c-fbd2c69373a1-serviceca\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.902277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901118 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-socket-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-cnibin\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-var-lib-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66348e6c-fe0c-43c0-989c-fbd2c69373a1-host\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/419a276e-db0b-4bbb-acbb-31859a30e77c-cni-binary-copy\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901246 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66348e6c-fe0c-43c0-989c-fbd2c69373a1-host\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901288 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-socket-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901293 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-device-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901331 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-ovn\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901337 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-device-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901356 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-run-ovn-kubernetes\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901370 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48059286-7d75-419d-b43e-f7a83ee4fdb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-env-overrides\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901383 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.903085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901370 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d03715cd-70aa-4174-9885-64293af4ef3f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.903879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.901454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-modprobe-d\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:19.904569 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.904549 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5991d28-df9c-45fe-a16b-dc6a4d1a25bd-agent-certs\") pod \"konnectivity-agent-bshpx\" (UID: \"b5991d28-df9c-45fe-a16b-dc6a4d1a25bd\") " pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:19.908458 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.908329 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:19.908458 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.908355 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:19.908458 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.908368 2580 projected.go:196] Error preparing data for projected volume kube-api-access-xzmwn for pod openshift-network-diagnostics/network-check-target-9j7b8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:19.908458 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:19.908442 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn podName:6741b66e-5e5c-4be3-8f51-4b1093928dc0 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:20.408405691 +0000 UTC m=+3.048247124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xzmwn" (UniqueName: "kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn") pod "network-check-target-9j7b8" (UID: "6741b66e-5e5c-4be3-8f51-4b1093928dc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:19.911269 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.911022 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg4bl\" (UniqueName: \"kubernetes.io/projected/48059286-7d75-419d-b43e-f7a83ee4fdb8-kube-api-access-kg4bl\") pod \"multus-additional-cni-plugins-9n5nt\" (UID: \"48059286-7d75-419d-b43e-f7a83ee4fdb8\") " pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:19.911269 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.911097 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbntw\" (UniqueName: \"kubernetes.io/projected/d03715cd-70aa-4174-9885-64293af4ef3f-kube-api-access-fbntw\") pod \"aws-ebs-csi-driver-node-sppcx\" (UID: \"d03715cd-70aa-4174-9885-64293af4ef3f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:19.911269 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.911135 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kjck\" (UniqueName: \"kubernetes.io/projected/bfd9e9d8-9609-43d5-a45c-e58816b43118-kube-api-access-9kjck\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:19.911269 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.911151 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbz4l\" (UniqueName: \"kubernetes.io/projected/f9581e35-486d-4e40-b4e7-96507433032c-kube-api-access-lbz4l\") pod \"iptables-alerter-js9c4\" (UID: \"f9581e35-486d-4e40-b4e7-96507433032c\") " pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:19.917409 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:19.917377 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg2s8\" (UniqueName: \"kubernetes.io/projected/66348e6c-fe0c-43c0-989c-fbd2c69373a1-kube-api-access-pg2s8\") pod \"node-ca-fhmb9\" (UID: \"66348e6c-fe0c-43c0-989c-fbd2c69373a1\") " pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:20.001966 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.001924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-var-lib-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.001966 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.001979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/419a276e-db0b-4bbb-acbb-31859a30e77c-cni-binary-copy\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.002191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002039 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-var-lib-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.002191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-ovn\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.002191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-run-ovn-kubernetes\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.002191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-env-overrides\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.002191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002142 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-modprobe-d\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.002191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-kubelet-config\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:20.002191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002181 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-dbus\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-cni-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-cni-bin\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002218 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-run-ovn-kubernetes\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-kubelet\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002218 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-ovn\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovn-node-metrics-cert\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002288 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-cni-bin\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002300 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwwbt\" (UniqueName: \"kubernetes.io/projected/0532dcdb-1611-488c-b8d3-6d84e84c687a-kube-api-access-qwwbt\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysctl-conf\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002324 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-modprobe-d\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002337 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-dbus\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002341 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-cni-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002380 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-hosts-file\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002385 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-kubelet\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002392 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-kubelet-config\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002451 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysctl-conf\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-system-cni-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.002521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-conf-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002537 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/419a276e-db0b-4bbb-acbb-31859a30e77c-cni-binary-copy\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-etc-kubernetes\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002574 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-log-socket\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002550 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-hosts-file\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002604 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-system-cni-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002651 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-conf-dir\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002673 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-env-overrides\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002685 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-log-socket\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002687 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-cni-bin\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002695 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-etc-kubernetes\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002734 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysconfig\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002740 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-cni-bin\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002803 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysconfig\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-tuned\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002877 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-os-release\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002960 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-etc-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.002999 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-os-release\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.003321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-cni-netd\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003082 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-etc-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003136 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-cni-netd\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovnkube-script-lib\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-host\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-hostroot\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003260 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-host\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003307 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-hostroot\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003332 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003337 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-openvswitch\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-netns\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-multus-certs\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.003406 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-slash\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-run\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003444 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-netns\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003461 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-multus-certs\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.004158 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.003468 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret podName:80821cae-47d3-482a-8fdd-ac42bcf3e4f7 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:20.503450704 +0000 UTC m=+3.143292130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret") pod "global-pull-secret-syncer-446pr" (UID: "80821cae-47d3-482a-8fdd-ac42bcf3e4f7") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003491 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-run\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003500 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-slash\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-kubelet\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-node-log\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003548 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-kubelet\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysctl-d\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003632 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-node-log\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003654 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovnkube-script-lib\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-daemon-config\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-kubernetes\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-systemd\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003722 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-sysctl-d\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovnkube-config\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-lib-modules\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003789 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-kubernetes\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003802 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-systemd\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-var-lib-kubelet\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.004872 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-tmp\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003850 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-tmp-dir\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w95ct\" (UniqueName: \"kubernetes.io/projected/419a276e-db0b-4bbb-acbb-31859a30e77c-kube-api-access-w95ct\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-systemd-units\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003891 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-lib-modules\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-run-netns\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-systemd\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-socket-dir-parent\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003990 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-k8s-cni-cncf-io\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-cni-multus\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004018 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-sys\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5m4r\" (UniqueName: \"kubernetes.io/projected/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-kube-api-access-p5m4r\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004067 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqpt\" (UniqueName: \"kubernetes.io/projected/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-kube-api-access-flqpt\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-cnibin\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004155 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-cnibin\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004167 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-daemon-config\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004188 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-run-netns\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.005638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004234 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-run-systemd\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004262 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-tmp-dir\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004269 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003938 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-var-lib-kubelet\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovnkube-config\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004312 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-run-k8s-cni-cncf-io\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004339 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-host-var-lib-cni-multus\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.003987 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0532dcdb-1611-488c-b8d3-6d84e84c687a-systemd-units\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004339 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/419a276e-db0b-4bbb-acbb-31859a30e77c-multus-socket-dir-parent\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.004456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-sys\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.005512 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-etc-tuned\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.006524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.005909 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0532dcdb-1611-488c-b8d3-6d84e84c687a-ovn-node-metrics-cert\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.007134 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.006608 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-tmp\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.012477 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.012406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwwbt\" (UniqueName: \"kubernetes.io/projected/0532dcdb-1611-488c-b8d3-6d84e84c687a-kube-api-access-qwwbt\") pod \"ovnkube-node-j9m6c\" (UID: \"0532dcdb-1611-488c-b8d3-6d84e84c687a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.013083 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.013002 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flqpt\" (UniqueName: \"kubernetes.io/projected/e2b2638a-0ff1-4b0f-9d68-10d8278f33a2-kube-api-access-flqpt\") pod \"node-resolver-w469r\" (UID: \"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2\") " pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:20.013496 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.013476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5m4r\" (UniqueName: \"kubernetes.io/projected/a70de4b5-5a90-45de-a1a4-9a78ef773ba1-kube-api-access-p5m4r\") pod \"tuned-wmwnv\" (UID: \"a70de4b5-5a90-45de-a1a4-9a78ef773ba1\") " pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.013737 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.013719 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w95ct\" (UniqueName: \"kubernetes.io/projected/419a276e-db0b-4bbb-acbb-31859a30e77c-kube-api-access-w95ct\") pod \"multus-d2jfx\" (UID: \"419a276e-db0b-4bbb-acbb-31859a30e77c\") " pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.093978 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.093875 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" Jun 08 12:07:20.101362 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.101337 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:20.110016 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.109996 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fhmb9" Jun 08 12:07:20.115549 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.115526 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" Jun 08 12:07:20.123036 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.123007 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-js9c4" Jun 08 12:07:20.129632 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.129616 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:20.137053 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.137036 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d2jfx" Jun 08 12:07:20.143525 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.143501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" Jun 08 12:07:20.148083 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.148067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w469r" Jun 08 12:07:20.406704 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.406633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:20.406842 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.406811 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:20.406896 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.406885 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:21.406863484 +0000 UTC m=+4.046704908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:20.507105 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.507068 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:20.507252 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.507147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:20.507252 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.507239 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:20.507347 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.507294 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret podName:80821cae-47d3-482a-8fdd-ac42bcf3e4f7 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:21.507280353 +0000 UTC m=+4.147121787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret") pod "global-pull-secret-syncer-446pr" (UID: "80821cae-47d3-482a-8fdd-ac42bcf3e4f7") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:20.507347 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.507239 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:20.507347 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.507344 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:20.507476 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.507359 2580 projected.go:196] Error preparing data for projected volume kube-api-access-xzmwn for pod openshift-network-diagnostics/network-check-target-9j7b8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:20.507476 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:20.507412 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn podName:6741b66e-5e5c-4be3-8f51-4b1093928dc0 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:21.50739215 +0000 UTC m=+4.147233566 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xzmwn" (UniqueName: "kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn") pod "network-check-target-9j7b8" (UID: "6741b66e-5e5c-4be3-8f51-4b1093928dc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:20.614629 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.614599 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b2638a_0ff1_4b0f_9d68_10d8278f33a2.slice/crio-88a1f15a2268a80aa7a2e2ecb9f5c116444b37ac90d3e2c89dcf5786389c1f90 WatchSource:0}: Error finding container 88a1f15a2268a80aa7a2e2ecb9f5c116444b37ac90d3e2c89dcf5786389c1f90: Status 404 returned error can't find the container with id 88a1f15a2268a80aa7a2e2ecb9f5c116444b37ac90d3e2c89dcf5786389c1f90 Jun 08 12:07:20.615933 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.615904 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd03715cd_70aa_4174_9885_64293af4ef3f.slice/crio-bc30aa53d6c641c28ece1797d32fa159ffdb2ab0c7fa7be5ec488cdf36d94727 WatchSource:0}: Error finding container bc30aa53d6c641c28ece1797d32fa159ffdb2ab0c7fa7be5ec488cdf36d94727: Status 404 returned error can't find the container with id bc30aa53d6c641c28ece1797d32fa159ffdb2ab0c7fa7be5ec488cdf36d94727 Jun 08 12:07:20.621299 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.621281 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70de4b5_5a90_45de_a1a4_9a78ef773ba1.slice/crio-ea68dc3f8255606b66be0c926e12a97362842f4fcad201ae24eb841a7d743fa3 WatchSource:0}: Error finding container ea68dc3f8255606b66be0c926e12a97362842f4fcad201ae24eb841a7d743fa3: Status 404 returned error can't find the container with id ea68dc3f8255606b66be0c926e12a97362842f4fcad201ae24eb841a7d743fa3 Jun 08 12:07:20.622953 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.622898 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48059286_7d75_419d_b43e_f7a83ee4fdb8.slice/crio-74979e8bd24987a4d69dc1b218a85b149515a9fcf85c6cfd99eee775d40f71ec WatchSource:0}: Error finding container 74979e8bd24987a4d69dc1b218a85b149515a9fcf85c6cfd99eee775d40f71ec: Status 404 returned error can't find the container with id 74979e8bd24987a4d69dc1b218a85b149515a9fcf85c6cfd99eee775d40f71ec Jun 08 12:07:20.623849 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.623731 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5991d28_df9c_45fe_a16b_dc6a4d1a25bd.slice/crio-90ea2cabfbe211f7dc699af9b43078086cfe66d7dc3cc50fddeccf4723ccc477 WatchSource:0}: Error finding container 90ea2cabfbe211f7dc699af9b43078086cfe66d7dc3cc50fddeccf4723ccc477: Status 404 returned error can't find the container with id 90ea2cabfbe211f7dc699af9b43078086cfe66d7dc3cc50fddeccf4723ccc477 Jun 08 12:07:20.625598 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.625579 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66348e6c_fe0c_43c0_989c_fbd2c69373a1.slice/crio-44ad9fab0239d73b7dd5bd5286a7d50d8b03efc09c9b5be5fe5c602111c9fc2b WatchSource:0}: Error finding container 44ad9fab0239d73b7dd5bd5286a7d50d8b03efc09c9b5be5fe5c602111c9fc2b: Status 404 returned error can't find the container with id 44ad9fab0239d73b7dd5bd5286a7d50d8b03efc09c9b5be5fe5c602111c9fc2b Jun 08 12:07:20.626583 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.626565 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419a276e_db0b_4bbb_acbb_31859a30e77c.slice/crio-aa03856bea08f0e7832be05e0a9f73af9e3ff5674ec3600d2ae41da330a00681 WatchSource:0}: Error finding container aa03856bea08f0e7832be05e0a9f73af9e3ff5674ec3600d2ae41da330a00681: Status 404 returned error can't find the container with id aa03856bea08f0e7832be05e0a9f73af9e3ff5674ec3600d2ae41da330a00681 Jun 08 12:07:20.627566 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.627277 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0532dcdb_1611_488c_b8d3_6d84e84c687a.slice/crio-0a4c3ee887e93d2f0453e259d5aebbac7df0714a5522b18f3d3c72dc31cb1af8 WatchSource:0}: Error finding container 0a4c3ee887e93d2f0453e259d5aebbac7df0714a5522b18f3d3c72dc31cb1af8: Status 404 returned error can't find the container with id 0a4c3ee887e93d2f0453e259d5aebbac7df0714a5522b18f3d3c72dc31cb1af8 Jun 08 12:07:20.628625 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:20.628235 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9581e35_486d_4e40_b4e7_96507433032c.slice/crio-9cb956105e0b21e2ad21c071910090b5d2cdf5ab832e686dcc469b85b89a7c1a WatchSource:0}: Error finding container 9cb956105e0b21e2ad21c071910090b5d2cdf5ab832e686dcc469b85b89a7c1a: Status 404 returned error can't find the container with id 9cb956105e0b21e2ad21c071910090b5d2cdf5ab832e686dcc469b85b89a7c1a Jun 08 12:07:20.660640 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.660459 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:20.828934 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.828900 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 12:02:19 +0000 UTC" deadline="2028-03-17 03:16:30.34270521 +0000 UTC" Jun 08 12:07:20.828934 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.828929 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15543h9m9.513779941s" Jun 08 12:07:20.862869 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.862839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerStarted","Data":"74979e8bd24987a4d69dc1b218a85b149515a9fcf85c6cfd99eee775d40f71ec"} Jun 08 12:07:20.863915 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.863894 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" event={"ID":"a70de4b5-5a90-45de-a1a4-9a78ef773ba1","Type":"ContainerStarted","Data":"ea68dc3f8255606b66be0c926e12a97362842f4fcad201ae24eb841a7d743fa3"} Jun 08 12:07:20.864932 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.864913 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" event={"ID":"d03715cd-70aa-4174-9885-64293af4ef3f","Type":"ContainerStarted","Data":"bc30aa53d6c641c28ece1797d32fa159ffdb2ab0c7fa7be5ec488cdf36d94727"} Jun 08 12:07:20.865903 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.865877 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w469r" event={"ID":"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2","Type":"ContainerStarted","Data":"88a1f15a2268a80aa7a2e2ecb9f5c116444b37ac90d3e2c89dcf5786389c1f90"} Jun 08 12:07:20.867711 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.867689 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" event={"ID":"ba29b9ee45638b00406cbe12817dc6fb","Type":"ContainerStarted","Data":"e89569205f87f61d5eefcc41525f7b127e625db3ee0d7a063a029f7b9006ff5a"} Jun 08 12:07:20.868844 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.868820 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"0a4c3ee887e93d2f0453e259d5aebbac7df0714a5522b18f3d3c72dc31cb1af8"} Jun 08 12:07:20.869682 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.869660 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fhmb9" event={"ID":"66348e6c-fe0c-43c0-989c-fbd2c69373a1","Type":"ContainerStarted","Data":"44ad9fab0239d73b7dd5bd5286a7d50d8b03efc09c9b5be5fe5c602111c9fc2b"} Jun 08 12:07:20.870671 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.870646 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-js9c4" event={"ID":"f9581e35-486d-4e40-b4e7-96507433032c","Type":"ContainerStarted","Data":"9cb956105e0b21e2ad21c071910090b5d2cdf5ab832e686dcc469b85b89a7c1a"} Jun 08 12:07:20.871871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.871846 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d2jfx" event={"ID":"419a276e-db0b-4bbb-acbb-31859a30e77c","Type":"ContainerStarted","Data":"aa03856bea08f0e7832be05e0a9f73af9e3ff5674ec3600d2ae41da330a00681"} Jun 08 12:07:20.873053 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.873017 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bshpx" event={"ID":"b5991d28-df9c-45fe-a16b-dc6a4d1a25bd","Type":"ContainerStarted","Data":"90ea2cabfbe211f7dc699af9b43078086cfe66d7dc3cc50fddeccf4723ccc477"} Jun 08 12:07:20.883277 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:20.883237 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" podStartSLOduration=1.883226959 podStartE2EDuration="1.883226959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:20.883191289 +0000 UTC m=+3.523032722" watchObservedRunningTime="2026-06-08 12:07:20.883226959 +0000 UTC m=+3.523068393" Jun 08 12:07:21.413818 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.413767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:21.414039 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.413977 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:21.414105 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.414041 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:23.414022504 +0000 UTC m=+6.053863940 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:21.515142 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.515103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:21.515343 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.515204 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:21.515434 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.515345 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:21.515434 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.515424 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret podName:80821cae-47d3-482a-8fdd-ac42bcf3e4f7 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:23.515388325 +0000 UTC m=+6.155229740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret") pod "global-pull-secret-syncer-446pr" (UID: "80821cae-47d3-482a-8fdd-ac42bcf3e4f7") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:21.516013 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.515666 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:21.516013 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.515694 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:21.516013 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.515708 2580 projected.go:196] Error preparing data for projected volume kube-api-access-xzmwn for pod openshift-network-diagnostics/network-check-target-9j7b8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:21.516013 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.515757 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn podName:6741b66e-5e5c-4be3-8f51-4b1093928dc0 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:23.515740808 +0000 UTC m=+6.155582222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xzmwn" (UniqueName: "kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn") pod "network-check-target-9j7b8" (UID: "6741b66e-5e5c-4be3-8f51-4b1093928dc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:21.829332 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.829268 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 12:02:19 +0000 UTC" deadline="2027-11-06 23:13:02.527588568 +0000 UTC" Jun 08 12:07:21.829332 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.829330 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12395h5m40.698262006s" Jun 08 12:07:21.857302 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.857274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:21.857473 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.857395 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:21.857814 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.857792 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:21.857925 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.857904 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:21.858010 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.857991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:21.858092 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:21.858075 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:21.888801 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.888768 2580 generic.go:356] "Generic (PLEG): container finished" podID="397bf72bf63ce09189ff4e95e7b44b6a" containerID="71d75e42b9b09b8fa4233309262c9fb187650c7169f6015ff0c541afbd1df9ff" exitCode=0 Jun 08 12:07:21.889677 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:21.889650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"397bf72bf63ce09189ff4e95e7b44b6a","Type":"ContainerDied","Data":"71d75e42b9b09b8fa4233309262c9fb187650c7169f6015ff0c541afbd1df9ff"} Jun 08 12:07:22.904341 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:22.904228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"397bf72bf63ce09189ff4e95e7b44b6a","Type":"ContainerStarted","Data":"3eabafffeba0e5dec76949b26601ac32aa8e4d4dc0d9bd0b5aa0134edfefcec5"} Jun 08 12:07:23.427429 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:23.427326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:23.427611 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.427454 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:23.427611 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.427527 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:27.427507598 +0000 UTC m=+10.067349015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:23.528588 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:23.528544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:23.528749 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:23.528615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:23.528820 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.528779 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:23.528881 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.528848 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret podName:80821cae-47d3-482a-8fdd-ac42bcf3e4f7 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:27.528827652 +0000 UTC m=+10.168669066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret") pod "global-pull-secret-syncer-446pr" (UID: "80821cae-47d3-482a-8fdd-ac42bcf3e4f7") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:23.535284 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.535121 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:23.535284 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.535160 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:23.535284 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.535174 2580 projected.go:196] Error preparing data for projected volume kube-api-access-xzmwn for pod openshift-network-diagnostics/network-check-target-9j7b8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:23.535284 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.535236 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn podName:6741b66e-5e5c-4be3-8f51-4b1093928dc0 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:27.535211491 +0000 UTC m=+10.175052904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xzmwn" (UniqueName: "kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn") pod "network-check-target-9j7b8" (UID: "6741b66e-5e5c-4be3-8f51-4b1093928dc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:23.855094 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:23.855056 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:23.855274 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.855212 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:23.855603 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:23.855580 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:23.855705 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.855684 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:23.855764 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:23.855740 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:23.855825 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:23.855803 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:25.854759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:25.854719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:25.854759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:25.854740 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:25.855272 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:25.854719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:25.855272 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:25.854835 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:25.855272 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:25.854929 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:25.855272 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:25.855039 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:27.461823 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:27.461789 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:27.462323 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.461985 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:27.462323 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.462048 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:35.462028391 +0000 UTC m=+18.101869808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:27.563149 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:27.563115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:27.563319 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:27.563174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:27.563319 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.563300 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:27.563432 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.563347 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:27.563432 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.563363 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret podName:80821cae-47d3-482a-8fdd-ac42bcf3e4f7 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:35.563345356 +0000 UTC m=+18.203186774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret") pod "global-pull-secret-syncer-446pr" (UID: "80821cae-47d3-482a-8fdd-ac42bcf3e4f7") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:27.563432 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.563369 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:27.563432 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.563380 2580 projected.go:196] Error preparing data for projected volume kube-api-access-xzmwn for pod openshift-network-diagnostics/network-check-target-9j7b8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:27.563432 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.563414 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn podName:6741b66e-5e5c-4be3-8f51-4b1093928dc0 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:35.56340434 +0000 UTC m=+18.203245762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xzmwn" (UniqueName: "kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn") pod "network-check-target-9j7b8" (UID: "6741b66e-5e5c-4be3-8f51-4b1093928dc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:27.864014 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:27.863716 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:27.864014 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:27.863909 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:27.864014 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:27.863969 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:27.864014 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.863954 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:27.864349 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.864106 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:27.864444 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:27.864333 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:29.854382 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:29.854341 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:29.854988 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:29.854354 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:29.854988 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:29.854501 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:29.854988 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:29.854577 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:29.854988 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:29.854357 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:29.854988 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:29.854690 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:31.854686 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:31.854656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:31.855149 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:31.854694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:31.855149 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:31.854657 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:31.855149 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:31.854795 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:31.855149 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:31.854891 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:31.855149 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:31.855012 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:33.855369 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:33.855337 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:33.855783 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:33.855337 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:33.855783 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:33.855445 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:33.855783 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:33.855456 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:33.855783 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:33.855519 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:33.855783 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:33.855627 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:35.523054 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:35.523016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:35.523559 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.523176 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:35.523559 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.523236 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:51.52322169 +0000 UTC m=+34.163063103 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:35.623302 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:35.623269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:35.623479 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:35.623333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:35.623479 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.623469 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:35.623600 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.623501 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:35.623600 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.623512 2580 projected.go:196] Error preparing data for projected volume kube-api-access-xzmwn for pod openshift-network-diagnostics/network-check-target-9j7b8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:35.623600 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.623474 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:35.623600 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.623564 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn podName:6741b66e-5e5c-4be3-8f51-4b1093928dc0 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:51.623544743 +0000 UTC m=+34.263386160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xzmwn" (UniqueName: "kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn") pod "network-check-target-9j7b8" (UID: "6741b66e-5e5c-4be3-8f51-4b1093928dc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:35.623600 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.623593 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret podName:80821cae-47d3-482a-8fdd-ac42bcf3e4f7 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:51.623574487 +0000 UTC m=+34.263415908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret") pod "global-pull-secret-syncer-446pr" (UID: "80821cae-47d3-482a-8fdd-ac42bcf3e4f7") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:35.854515 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:35.854418 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:35.854515 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:35.854432 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:35.854722 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.854583 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:35.854722 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:35.854632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:35.854722 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.854711 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:35.855054 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:35.855020 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:37.855018 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:37.854988 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:37.855533 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:37.855101 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:37.855533 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:37.855102 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:37.855533 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:37.855136 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:37.855533 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:37.855207 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:37.855533 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:37.855287 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:39.856026 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.855785 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:39.856504 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:39.856143 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:39.856504 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.855874 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:39.856504 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:39.856239 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:39.856659 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.855785 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:39.856659 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:39.856646 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:39.938258 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.938225 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" event={"ID":"a70de4b5-5a90-45de-a1a4-9a78ef773ba1","Type":"ContainerStarted","Data":"b0d7fdca7d0d3a4065747a3fff254a82e57e69409381efbdfd375c19fd662db6"} Jun 08 12:07:39.939533 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.939509 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" event={"ID":"d03715cd-70aa-4174-9885-64293af4ef3f","Type":"ContainerStarted","Data":"399c28cecbfe97f819949dda301760465c2113e17021150dbc1324f7cd1be88d"} Jun 08 12:07:39.940784 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.940759 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w469r" event={"ID":"e2b2638a-0ff1-4b0f-9d68-10d8278f33a2","Type":"ContainerStarted","Data":"1f707781a4a5f820ebbc7e27d2b04d210e6ebb4e59d21ffe43b13e7d77f7fa1d"} Jun 08 12:07:39.942208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.942183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fhmb9" event={"ID":"66348e6c-fe0c-43c0-989c-fbd2c69373a1","Type":"ContainerStarted","Data":"0aed5ba30e8a05189026e80d1287f7836d947c67800ba833f8e3fe71c3680869"} Jun 08 12:07:39.943549 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.943528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d2jfx" event={"ID":"419a276e-db0b-4bbb-acbb-31859a30e77c","Type":"ContainerStarted","Data":"f6d886f0b14ee8b7e1b61298071b1b09faf3b072711e0a48065e903af6ecb49b"} Jun 08 12:07:39.944852 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.944825 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bshpx" event={"ID":"b5991d28-df9c-45fe-a16b-dc6a4d1a25bd","Type":"ContainerStarted","Data":"7ec8a7c3f17d731f5fb5d2b4c0c238f96b43d6ec184c032fbc77c56a705cead2"} Jun 08 12:07:39.946070 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.946049 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerStarted","Data":"4e0bb7dd02791417d85396683a72b0db7c9cbada8a5dac157921a67ce445305d"} Jun 08 12:07:39.955565 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.955524 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wmwnv" podStartSLOduration=3.129367934 podStartE2EDuration="21.955512652s" totalImagesPullingTime="18.826144718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:39.955161753 +0000 UTC m=+22.595003188" watchObservedRunningTime="2026-06-08 12:07:39.955512652 +0000 UTC m=+22.595354105" Jun 08 12:07:39.955705 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.955681 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" podStartSLOduration=20.955673557 podStartE2EDuration="20.955673557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:22.924155891 +0000 UTC m=+5.563997325" watchObservedRunningTime="2026-06-08 12:07:39.955673557 +0000 UTC m=+22.595514991" Jun 08 12:07:39.969250 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:39.969207 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fhmb9" podStartSLOduration=3.126233276 podStartE2EDuration="21.969193051s" totalImagesPullingTime="18.842959775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:39.969172815 +0000 UTC m=+22.609014250" watchObservedRunningTime="2026-06-08 12:07:39.969193051 +0000 UTC m=+22.609034485" Jun 08 12:07:40.007138 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.006331 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w469r" podStartSLOduration=3.145321509 podStartE2EDuration="22.006311041s" totalImagesPullingTime="18.860989532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:40.006150422 +0000 UTC m=+22.645991866" watchObservedRunningTime="2026-06-08 12:07:40.006311041 +0000 UTC m=+22.646152478" Jun 08 12:07:40.007138 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.006443 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d2jfx" podStartSLOduration=3.125179789 podStartE2EDuration="22.006436868s" totalImagesPullingTime="18.881257079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:39.987426119 +0000 UTC m=+22.627267554" watchObservedRunningTime="2026-06-08 12:07:40.006436868 +0000 UTC m=+22.646278301" Jun 08 12:07:40.045937 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.045860 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bshpx" podStartSLOduration=4.221525742 podStartE2EDuration="23.045838038s" totalImagesPullingTime="18.824312296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:40.044782912 +0000 UTC m=+22.684624359" watchObservedRunningTime="2026-06-08 12:07:40.045838038 +0000 UTC m=+22.685679479" Jun 08 12:07:40.949355 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.949273 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-js9c4" event={"ID":"f9581e35-486d-4e40-b4e7-96507433032c","Type":"ContainerStarted","Data":"663f5fd3ff29cefa51db103067eb2b834a8fcd6440cf62cca3b6409fe9586903"} Jun 08 12:07:40.950705 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.950681 2580 generic.go:356] "Generic (PLEG): container finished" podID="48059286-7d75-419d-b43e-f7a83ee4fdb8" containerID="4e0bb7dd02791417d85396683a72b0db7c9cbada8a5dac157921a67ce445305d" exitCode=0 Jun 08 12:07:40.950824 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.950762 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerDied","Data":"4e0bb7dd02791417d85396683a72b0db7c9cbada8a5dac157921a67ce445305d"} Jun 08 12:07:40.953315 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.953300 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:07:40.953598 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.953581 2580 generic.go:356] "Generic (PLEG): container finished" podID="0532dcdb-1611-488c-b8d3-6d84e84c687a" containerID="f4ed8662a0301d468768c6e0fc43cf9e9de2e9b9a648a52e609925c1f73eaf47" exitCode=1 Jun 08 12:07:40.953705 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.953686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"7011cd77e481a1dd30fa276d746441790ced8783f96b46eb4c49def2c0a412d6"} Jun 08 12:07:40.953793 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.953779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"97a581b07760c9dec00116f4678076fbc65bb55eee6f9e2b93aa4317d9d00bc2"} Jun 08 12:07:40.953846 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.953835 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"5d90bc8cbdc08754d5243eb43386351e053eec7a37c0ea180023d567d12f9a88"} Jun 08 12:07:40.953906 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.953892 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"8229f3f145b95ccf20e4d7e1ecc8c9849f666550ad624f320073699b9dd31141"} Jun 08 12:07:40.953975 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.953962 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerDied","Data":"f4ed8662a0301d468768c6e0fc43cf9e9de2e9b9a648a52e609925c1f73eaf47"} Jun 08 12:07:40.954053 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.954039 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"53a68a618b0c0f19ea1185780e8e4f4640dc57e8ad715b6e6469b01583dbc81c"} Jun 08 12:07:40.965499 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:40.965464 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-js9c4" podStartSLOduration=4.145703868 podStartE2EDuration="22.965451627s" totalImagesPullingTime="18.819747759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:40.965257511 +0000 UTC m=+23.605098947" watchObservedRunningTime="2026-06-08 12:07:40.965451627 +0000 UTC m=+23.605293061" Jun 08 12:07:41.246232 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:41.246210 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 08 12:07:41.854717 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:41.854683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:41.854717 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:41.854701 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:41.855148 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:41.854692 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:41.855148 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:41.854847 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:41.855148 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:41.854890 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:41.855148 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:41.854993 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:41.861019 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:41.860925 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-08T12:07:41.2462275Z","UUID":"74072e87-5e9a-4a32-bad2-e6a5bf8a6860","Handler":null,"Name":"","Endpoint":""} Jun 08 12:07:41.862636 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:41.862618 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 08 12:07:41.862744 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:41.862658 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 08 12:07:41.957958 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:41.957905 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" event={"ID":"d03715cd-70aa-4174-9885-64293af4ef3f","Type":"ContainerStarted","Data":"4af6ff874a04f0fecb36e74391dd81820126bf0609da4c42a0b8d5d5a2d46cb6"} Jun 08 12:07:42.323124 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:42.323093 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:42.323812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:42.323790 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:42.963430 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:42.963299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" event={"ID":"d03715cd-70aa-4174-9885-64293af4ef3f","Type":"ContainerStarted","Data":"281869be51393c0fb281ee138be95bb7bb9bc8afc6c2787df69c7ba0fb6d61de"} Jun 08 12:07:42.967237 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:42.967215 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:07:42.967654 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:42.967632 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"0aad1036a81f0f27ff7f8eb5383c3575e47e5ee0dd8f7d31879f98a5c699535c"} Jun 08 12:07:42.981912 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:42.981862 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sppcx" podStartSLOduration=3.59154244 podStartE2EDuration="24.981847136s" totalImagesPullingTime="21.390304696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:42.981801919 +0000 UTC m=+25.621643353" watchObservedRunningTime="2026-06-08 12:07:42.981847136 +0000 UTC m=+25.621688549" Jun 08 12:07:43.855274 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:43.855244 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:43.855458 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:43.855253 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:43.855458 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:43.855348 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:43.855458 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:43.855253 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:43.855458 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:43.855431 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:43.855671 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:43.855527 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:45.854435 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.854277 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:45.854886 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.854278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:45.854886 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:45.854527 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:45.854886 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.854278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:45.854886 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:45.854627 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:45.854886 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:45.854690 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:45.976284 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.976253 2580 generic.go:356] "Generic (PLEG): container finished" podID="48059286-7d75-419d-b43e-f7a83ee4fdb8" containerID="56008fdc29965fc318f4a31400ae5523cc5fba70416ad804e96b3d825726cbf3" exitCode=0 Jun 08 12:07:45.976394 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.976334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerDied","Data":"56008fdc29965fc318f4a31400ae5523cc5fba70416ad804e96b3d825726cbf3"} Jun 08 12:07:45.979971 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.979938 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:07:45.980326 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.980299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"ecca1820d40acfad80a820d628a4646d130ce37c119e2bef852f43a7ed8ddd88"} Jun 08 12:07:45.980537 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.980514 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:45.980652 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.980599 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:45.980652 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.980642 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:45.980764 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.980725 2580 scope.go:117] "RemoveContainer" containerID="f4ed8662a0301d468768c6e0fc43cf9e9de2e9b9a648a52e609925c1f73eaf47" Jun 08 12:07:45.996476 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.996412 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:45.997109 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:45.997091 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:07:46.986435 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:46.986408 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:07:46.986886 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:46.986759 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" event={"ID":"0532dcdb-1611-488c-b8d3-6d84e84c687a","Type":"ContainerStarted","Data":"27d6995ec52a0c1f24bb813d7c711a83abc6503773cfa0d2f8325f46d881375b"} Jun 08 12:07:47.017048 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.016999 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" podStartSLOduration=9.647923063 podStartE2EDuration="29.01697548s" totalImagesPullingTime="19.369052417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:47.015137192 +0000 UTC m=+29.654978660" watchObservedRunningTime="2026-06-08 12:07:47.01697548 +0000 UTC m=+29.656816914" Jun 08 12:07:47.449902 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.449707 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-446pr"] Jun 08 12:07:47.450176 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.450008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:47.450176 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:47.450112 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:47.450712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.450450 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wr6xl"] Jun 08 12:07:47.450712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.450559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:47.450712 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:47.450660 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:47.451213 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.451184 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9j7b8"] Jun 08 12:07:47.451323 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.451274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:47.451387 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:47.451365 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:47.885349 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.885320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:47.980801 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.980775 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:47.992934 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.992907 2580 generic.go:356] "Generic (PLEG): container finished" podID="48059286-7d75-419d-b43e-f7a83ee4fdb8" containerID="09e7dc0d8a311be3aa302cebfef844cd49aca3be1e05d2c5a54bec885c2eb4e3" exitCode=0 Jun 08 12:07:47.993383 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:47.992998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerDied","Data":"09e7dc0d8a311be3aa302cebfef844cd49aca3be1e05d2c5a54bec885c2eb4e3"} Jun 08 12:07:48.095784 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.095750 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:48.169614 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.169537 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:48.268170 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.268140 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:48.438628 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.438554 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:48.438772 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.438673 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 08 12:07:48.439065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.439052 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bshpx" Jun 08 12:07:48.505575 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.505542 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:48.855033 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.855001 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:48.855214 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:48.855007 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:48.855214 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:48.855107 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:48.855214 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:48.855179 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:49.166096 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:49.166065 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:49.266825 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:49.266801 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:49.854574 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:49.854494 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:49.854697 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:49.854633 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:49.999262 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:49.999228 2580 generic.go:356] "Generic (PLEG): container finished" podID="48059286-7d75-419d-b43e-f7a83ee4fdb8" containerID="d0325782dd0aa57d2a9e9896dcbcb76ffd918147c60d89df18df8f61b5ebbf26" exitCode=0 Jun 08 12:07:49.999439 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:49.999289 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerDied","Data":"d0325782dd0aa57d2a9e9896dcbcb76ffd918147c60d89df18df8f61b5ebbf26"} Jun 08 12:07:50.855093 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:50.855063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:50.855769 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:50.855063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:50.855769 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:50.855170 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-446pr" podUID="80821cae-47d3-482a-8fdd-ac42bcf3e4f7" Jun 08 12:07:50.855769 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:50.855232 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j7b8" podUID="6741b66e-5e5c-4be3-8f51-4b1093928dc0" Jun 08 12:07:51.446168 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:51.446137 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:51.546326 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:51.546288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:51.555225 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:51.555193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:51.555366 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.555305 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:51.555366 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.555366 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:23.555348636 +0000 UTC m=+66.195190050 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:51.655932 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:51.655893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:51.656121 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:51.655979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:51.656121 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.656061 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:51.656121 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.656081 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:51.656121 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.656084 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:51.656121 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.656093 2580 projected.go:196] Error preparing data for projected volume kube-api-access-xzmwn for pod openshift-network-diagnostics/network-check-target-9j7b8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:51.656376 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.656143 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn podName:6741b66e-5e5c-4be3-8f51-4b1093928dc0 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:23.656127403 +0000 UTC m=+66.295968836 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xzmwn" (UniqueName: "kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn") pod "network-check-target-9j7b8" (UID: "6741b66e-5e5c-4be3-8f51-4b1093928dc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:51.656376 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.656159 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret podName:80821cae-47d3-482a-8fdd-ac42bcf3e4f7 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:23.656153485 +0000 UTC m=+66.295994900 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret") pod "global-pull-secret-syncer-446pr" (UID: "80821cae-47d3-482a-8fdd-ac42bcf3e4f7") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:51.806164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:51.806127 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:51.854620 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:51.854592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:51.854789 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:51.854725 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:07:51.926561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:51.926528 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:52.627924 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.627853 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeReady" Jun 08 12:07:52.628095 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.628011 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 08 12:07:52.674038 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.673979 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8gqn6"] Jun 08 12:07:52.694098 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.694073 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5rmvq"] Jun 08 12:07:52.694246 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.694203 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.697028 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.697006 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6km9q\"" Jun 08 12:07:52.697170 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.697006 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 08 12:07:52.697170 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.697010 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 08 12:07:52.716196 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.716173 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8gqn6"] Jun 08 12:07:52.716330 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.716215 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5rmvq"] Jun 08 12:07:52.716330 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.716316 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5rmvq" Jun 08 12:07:52.719333 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.719307 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 08 12:07:52.719333 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.719328 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2hkrn\"" Jun 08 12:07:52.719611 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.719593 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 08 12:07:52.719737 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.719719 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 08 12:07:52.811696 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.811666 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:52.854749 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.854720 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:07:52.854896 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.854729 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:07:52.857759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.857730 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 12:07:52.857759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.857745 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 12:07:52.857759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.857760 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 12:07:52.857966 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.857925 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qfftc\"" Jun 08 12:07:52.865061 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.865032 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:52.867652 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.867628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/636ec464-e603-4917-a35e-cdfa55a4491b-cert\") pod \"ingress-canary-5rmvq\" (UID: \"636ec464-e603-4917-a35e-cdfa55a4491b\") " pod="openshift-ingress-canary/ingress-canary-5rmvq" Jun 08 12:07:52.867742 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.867689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.867742 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.867716 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55247bf6-9e28-41a7-b925-b8e43914ef45-tmp-dir\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.867857 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.867766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm5hk\" (UniqueName: \"kubernetes.io/projected/636ec464-e603-4917-a35e-cdfa55a4491b-kube-api-access-qm5hk\") pod \"ingress-canary-5rmvq\" (UID: \"636ec464-e603-4917-a35e-cdfa55a4491b\") " pod="openshift-ingress-canary/ingress-canary-5rmvq" Jun 08 12:07:52.867857 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.867834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtv5l\" (UniqueName: \"kubernetes.io/projected/55247bf6-9e28-41a7-b925-b8e43914ef45-kube-api-access-qtv5l\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.867973 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.867874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55247bf6-9e28-41a7-b925-b8e43914ef45-config-volume\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.968459 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.968400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/636ec464-e603-4917-a35e-cdfa55a4491b-cert\") pod \"ingress-canary-5rmvq\" (UID: \"636ec464-e603-4917-a35e-cdfa55a4491b\") " pod="openshift-ingress-canary/ingress-canary-5rmvq" Jun 08 12:07:52.968459 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.968441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.968910 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.968475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55247bf6-9e28-41a7-b925-b8e43914ef45-tmp-dir\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.968910 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.968495 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm5hk\" (UniqueName: \"kubernetes.io/projected/636ec464-e603-4917-a35e-cdfa55a4491b-kube-api-access-qm5hk\") pod \"ingress-canary-5rmvq\" (UID: \"636ec464-e603-4917-a35e-cdfa55a4491b\") " pod="openshift-ingress-canary/ingress-canary-5rmvq" Jun 08 12:07:52.968910 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.968545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtv5l\" (UniqueName: \"kubernetes.io/projected/55247bf6-9e28-41a7-b925-b8e43914ef45-kube-api-access-qtv5l\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.968910 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.968575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55247bf6-9e28-41a7-b925-b8e43914ef45-config-volume\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.968910 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.968842 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55247bf6-9e28-41a7-b925-b8e43914ef45-tmp-dir\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.968910 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:52.968870 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:52.969211 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:52.968928 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:53.468910599 +0000 UTC m=+36.108752026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:07:52.969211 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.969143 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55247bf6-9e28-41a7-b925-b8e43914ef45-config-volume\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.973572 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.973381 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/636ec464-e603-4917-a35e-cdfa55a4491b-cert\") pod \"ingress-canary-5rmvq\" (UID: \"636ec464-e603-4917-a35e-cdfa55a4491b\") " pod="openshift-ingress-canary/ingress-canary-5rmvq" Jun 08 12:07:52.979349 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.979321 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtv5l\" (UniqueName: \"kubernetes.io/projected/55247bf6-9e28-41a7-b925-b8e43914ef45-kube-api-access-qtv5l\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:52.979586 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:52.979567 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm5hk\" (UniqueName: \"kubernetes.io/projected/636ec464-e603-4917-a35e-cdfa55a4491b-kube-api-access-qm5hk\") pod \"ingress-canary-5rmvq\" (UID: \"636ec464-e603-4917-a35e-cdfa55a4491b\") " pod="openshift-ingress-canary/ingress-canary-5rmvq" Jun 08 12:07:53.026281 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:53.026256 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5rmvq" Jun 08 12:07:53.109348 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:53.109320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:53.206803 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:53.206770 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5rmvq"] Jun 08 12:07:53.473183 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:53.473138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:53.473374 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:53.473330 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:53.473446 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:53.473434 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:54.473411986 +0000 UTC m=+37.113253403 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:07:53.854957 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:53.854914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:07:53.858129 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:53.858102 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 12:07:53.858423 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:53.858394 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gr2v8\"" Jun 08 12:07:54.480704 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:54.480667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:54.481136 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:54.480844 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:54.481136 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:54.480920 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:56.480904886 +0000 UTC m=+39.120746298 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:07:54.675580 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:54.675551 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636ec464_e603_4917_a35e_cdfa55a4491b.slice/crio-ce7bfa59ff961773ba42cd93b81d37f81115f102110989f573deb88cf489ebe3 WatchSource:0}: Error finding container ce7bfa59ff961773ba42cd93b81d37f81115f102110989f573deb88cf489ebe3: Status 404 returned error can't find the container with id ce7bfa59ff961773ba42cd93b81d37f81115f102110989f573deb88cf489ebe3 Jun 08 12:07:55.013003 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:55.012969 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5rmvq" event={"ID":"636ec464-e603-4917-a35e-cdfa55a4491b","Type":"ContainerStarted","Data":"ce7bfa59ff961773ba42cd93b81d37f81115f102110989f573deb88cf489ebe3"} Jun 08 12:07:56.018347 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:56.018310 2580 generic.go:356] "Generic (PLEG): container finished" podID="48059286-7d75-419d-b43e-f7a83ee4fdb8" containerID="06bb3c547495f14159ee88c6fdc1b523c3c0070ac54729f6e4798ed36bafddc8" exitCode=0 Jun 08 12:07:56.018800 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:56.018359 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerDied","Data":"06bb3c547495f14159ee88c6fdc1b523c3c0070ac54729f6e4798ed36bafddc8"} Jun 08 12:07:56.470696 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:56.470672 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:56.495144 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:56.495119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:07:56.495261 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:56.495223 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:56.495301 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:07:56.495275 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:00.495258253 +0000 UTC m=+43.135099668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:07:56.568070 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:56.567998 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:57.022286 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.022246 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5rmvq" event={"ID":"636ec464-e603-4917-a35e-cdfa55a4491b","Type":"ContainerStarted","Data":"83a2c354794c65218a778b437440394ba957531c22a397375c55e8c2355a3f94"} Jun 08 12:07:57.025176 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.025153 2580 generic.go:356] "Generic (PLEG): container finished" podID="48059286-7d75-419d-b43e-f7a83ee4fdb8" containerID="a6bf77d81a1f8285a7a74456cac6fe5e895602f42122ccce8833ff84077c720e" exitCode=0 Jun 08 12:07:57.025286 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.025186 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerDied","Data":"a6bf77d81a1f8285a7a74456cac6fe5e895602f42122ccce8833ff84077c720e"} Jun 08 12:07:57.067726 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.067664 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5rmvq" podStartSLOduration=3.358686809 podStartE2EDuration="5.06764198s" totalImagesPullingTime="1.708955171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:57.040558949 +0000 UTC m=+39.680400385" watchObservedRunningTime="2026-06-08 12:07:57.06764198 +0000 UTC m=+39.707483415" Jun 08 12:07:57.289969 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.289890 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv"] Jun 08 12:07:57.305160 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.305134 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv"] Jun 08 12:07:57.305261 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.305222 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:07:57.307876 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.307855 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 08 12:07:57.308005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.307882 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 08 12:07:57.308005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.307891 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-cx8qz\"" Jun 08 12:07:57.308005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.307886 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 08 12:07:57.308241 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.308226 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 08 12:07:57.318220 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.318192 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf"] Jun 08 12:07:57.337246 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.337227 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf"] Jun 08 12:07:57.337324 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.337309 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.339843 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.339824 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 08 12:07:57.402362 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.402333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z64k7\" (UniqueName: \"kubernetes.io/projected/fdecc9cb-450e-41a7-a3b1-4f656f24fed0-kube-api-access-z64k7\") pod \"managed-serviceaccount-addon-agent-7f8774f974-76zgv\" (UID: \"fdecc9cb-450e-41a7-a3b1-4f656f24fed0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:07:57.402485 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.402442 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fdecc9cb-450e-41a7-a3b1-4f656f24fed0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f8774f974-76zgv\" (UID: \"fdecc9cb-450e-41a7-a3b1-4f656f24fed0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:07:57.503456 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.503421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fdecc9cb-450e-41a7-a3b1-4f656f24fed0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f8774f974-76zgv\" (UID: \"fdecc9cb-450e-41a7-a3b1-4f656f24fed0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:07:57.503456 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.503462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9ae6b08f-8528-497b-afbc-83f62e504b14-klusterlet-config\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.503641 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.503484 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz25s\" (UniqueName: \"kubernetes.io/projected/9ae6b08f-8528-497b-afbc-83f62e504b14-kube-api-access-gz25s\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.503641 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.503516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z64k7\" (UniqueName: \"kubernetes.io/projected/fdecc9cb-450e-41a7-a3b1-4f656f24fed0-kube-api-access-z64k7\") pod \"managed-serviceaccount-addon-agent-7f8774f974-76zgv\" (UID: \"fdecc9cb-450e-41a7-a3b1-4f656f24fed0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:07:57.503641 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.503550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ae6b08f-8528-497b-afbc-83f62e504b14-tmp\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.507019 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.506998 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fdecc9cb-450e-41a7-a3b1-4f656f24fed0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f8774f974-76zgv\" (UID: \"fdecc9cb-450e-41a7-a3b1-4f656f24fed0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:07:57.511910 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.511890 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z64k7\" (UniqueName: \"kubernetes.io/projected/fdecc9cb-450e-41a7-a3b1-4f656f24fed0-kube-api-access-z64k7\") pod \"managed-serviceaccount-addon-agent-7f8774f974-76zgv\" (UID: \"fdecc9cb-450e-41a7-a3b1-4f656f24fed0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:07:57.604111 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.604054 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9ae6b08f-8528-497b-afbc-83f62e504b14-klusterlet-config\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.604111 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.604083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz25s\" (UniqueName: \"kubernetes.io/projected/9ae6b08f-8528-497b-afbc-83f62e504b14-kube-api-access-gz25s\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.604111 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.604111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ae6b08f-8528-497b-afbc-83f62e504b14-tmp\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.604415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.604399 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ae6b08f-8528-497b-afbc-83f62e504b14-tmp\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.606378 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.606355 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9ae6b08f-8528-497b-afbc-83f62e504b14-klusterlet-config\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.612577 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.612555 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz25s\" (UniqueName: \"kubernetes.io/projected/9ae6b08f-8528-497b-afbc-83f62e504b14-kube-api-access-gz25s\") pod \"klusterlet-addon-workmgr-fc8bb9cf7-mjrcf\" (UID: \"9ae6b08f-8528-497b-afbc-83f62e504b14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.621542 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.621526 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:07:57.645624 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.645604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:07:57.775567 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.775392 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv"] Jun 08 12:07:57.778930 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:57.778907 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdecc9cb_450e_41a7_a3b1_4f656f24fed0.slice/crio-e3a2ec2ff5377088ca70539023570e2948e27775464a073747bc49a15209ff11 WatchSource:0}: Error finding container e3a2ec2ff5377088ca70539023570e2948e27775464a073747bc49a15209ff11: Status 404 returned error can't find the container with id e3a2ec2ff5377088ca70539023570e2948e27775464a073747bc49a15209ff11 Jun 08 12:07:57.793483 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.793461 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf"] Jun 08 12:07:57.796179 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:07:57.796154 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae6b08f_8528_497b_afbc_83f62e504b14.slice/crio-f0eee6f8b3b971150d29a0f7dc7011f9a31f7b3c218a5fbd6f6a2cd60778d1df WatchSource:0}: Error finding container f0eee6f8b3b971150d29a0f7dc7011f9a31f7b3c218a5fbd6f6a2cd60778d1df: Status 404 returned error can't find the container with id f0eee6f8b3b971150d29a0f7dc7011f9a31f7b3c218a5fbd6f6a2cd60778d1df Jun 08 12:07:57.816137 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.816115 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:57.917130 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:57.917056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:58.028065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:58.028034 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" event={"ID":"fdecc9cb-450e-41a7-a3b1-4f656f24fed0","Type":"ContainerStarted","Data":"e3a2ec2ff5377088ca70539023570e2948e27775464a073747bc49a15209ff11"} Jun 08 12:07:58.030888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:58.030859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" event={"ID":"48059286-7d75-419d-b43e-f7a83ee4fdb8","Type":"ContainerStarted","Data":"599f7773f7f5db26490a09c9e12ae636afc8774b8441c9e884386c4ae7b269bc"} Jun 08 12:07:58.031836 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:58.031814 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" event={"ID":"9ae6b08f-8528-497b-afbc-83f62e504b14","Type":"ContainerStarted","Data":"f0eee6f8b3b971150d29a0f7dc7011f9a31f7b3c218a5fbd6f6a2cd60778d1df"} Jun 08 12:07:58.241305 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:58.241269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:58.299694 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:58.299663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:07:58.370982 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:07:58.370926 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:00.362106 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:00.362045 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:00.462804 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:00.462774 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:00.527003 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:00.526976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:08:00.527179 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:00.527085 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:08:00.527179 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:00.527139 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:08.527123046 +0000 UTC m=+51.166964472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:08:01.041615 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:01.041562 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" event={"ID":"fdecc9cb-450e-41a7-a3b1-4f656f24fed0","Type":"ContainerStarted","Data":"434ee0b034dda347a31fb02ac463af3f6756845838220eef49fe181a52e68285"} Jun 08 12:08:01.057101 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:01.057001 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" podStartSLOduration=1.264870799 podStartE2EDuration="4.056984838s" totalImagesPullingTime="2.792114039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:08:01.056254627 +0000 UTC m=+43.696096081" watchObservedRunningTime="2026-06-08 12:08:01.056984838 +0000 UTC m=+43.696826272" Jun 08 12:08:01.057344 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:01.057312 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9n5nt" podStartSLOduration=13.204685902 podStartE2EDuration="44.0573026s" totalImagesPullingTime="30.852616698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:58.065016915 +0000 UTC m=+40.704858350" watchObservedRunningTime="2026-06-08 12:08:01.0573026 +0000 UTC m=+43.697144035" Jun 08 12:08:04.051463 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:04.051422 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" event={"ID":"9ae6b08f-8528-497b-afbc-83f62e504b14","Type":"ContainerStarted","Data":"07c042bba791c5b67994a23a039f05a10da0d11f574325d7ee054fb1953f8e28"} Jun 08 12:08:04.051897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:04.051617 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:08:04.053092 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:04.053073 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:08:04.067776 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:04.067734 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" podStartSLOduration=0.958014317 podStartE2EDuration="7.067721775s" totalImagesPullingTime="6.109707458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:08:04.067142039 +0000 UTC m=+46.706983475" watchObservedRunningTime="2026-06-08 12:08:04.067721775 +0000 UTC m=+46.707563210" Jun 08 12:08:05.104400 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:05.104365 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:05.205290 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:05.205257 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:05.306230 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:05.306194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:05.360624 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:05.360556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:08.586898 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:08.586868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:08:08.587332 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:08.587024 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:08:08.587332 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:08.587077 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:24.587063212 +0000 UTC m=+67.226904628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:08:12.661626 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:12.661593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:12.728694 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:12.728664 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:18.002602 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:18.002577 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9m6c" Jun 08 12:08:18.152764 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:18.152716 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:18.253421 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:18.253319 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:23.583385 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.583352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:08:23.586271 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.586254 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 12:08:23.594401 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:23.594385 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 12:08:23.594447 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:23.594438 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:09:27.594422526 +0000 UTC m=+130.234263938 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : secret "metrics-daemon-secret" not found Jun 08 12:08:23.684296 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.684269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:08:23.684413 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.684329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:08:23.686978 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.686958 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 12:08:23.687091 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.687020 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 12:08:23.697066 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.697045 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 12:08:23.697791 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.697773 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80821cae-47d3-482a-8fdd-ac42bcf3e4f7-original-pull-secret\") pod \"global-pull-secret-syncer-446pr\" (UID: \"80821cae-47d3-482a-8fdd-ac42bcf3e4f7\") " pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:08:23.707540 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.707521 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzmwn\" (UniqueName: \"kubernetes.io/projected/6741b66e-5e5c-4be3-8f51-4b1093928dc0-kube-api-access-xzmwn\") pod \"network-check-target-9j7b8\" (UID: \"6741b66e-5e5c-4be3-8f51-4b1093928dc0\") " pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:08:23.769224 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.769200 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qfftc\"" Jun 08 12:08:23.771324 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.771309 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-446pr" Jun 08 12:08:23.776795 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.776770 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:08:23.901742 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.901707 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-446pr"] Jun 08 12:08:23.906053 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:08:23.906024 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80821cae_47d3_482a_8fdd_ac42bcf3e4f7.slice/crio-7fe8d29f78119b950dfd562f5ef7e4317d9c2158a5b137b8a4402fcb8c68679c WatchSource:0}: Error finding container 7fe8d29f78119b950dfd562f5ef7e4317d9c2158a5b137b8a4402fcb8c68679c: Status 404 returned error can't find the container with id 7fe8d29f78119b950dfd562f5ef7e4317d9c2158a5b137b8a4402fcb8c68679c Jun 08 12:08:23.918660 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:23.918637 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9j7b8"] Jun 08 12:08:23.921576 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:08:23.921548 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6741b66e_5e5c_4be3_8f51_4b1093928dc0.slice/crio-baf4853f9aea7e5b0abfa17c1f4fd9aafe6d0ade4152b5baf51488b51194fc66 WatchSource:0}: Error finding container baf4853f9aea7e5b0abfa17c1f4fd9aafe6d0ade4152b5baf51488b51194fc66: Status 404 returned error can't find the container with id baf4853f9aea7e5b0abfa17c1f4fd9aafe6d0ade4152b5baf51488b51194fc66 Jun 08 12:08:24.100478 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:24.100443 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9j7b8" event={"ID":"6741b66e-5e5c-4be3-8f51-4b1093928dc0","Type":"ContainerStarted","Data":"baf4853f9aea7e5b0abfa17c1f4fd9aafe6d0ade4152b5baf51488b51194fc66"} Jun 08 12:08:24.101480 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:24.101456 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-446pr" event={"ID":"80821cae-47d3-482a-8fdd-ac42bcf3e4f7","Type":"ContainerStarted","Data":"7fe8d29f78119b950dfd562f5ef7e4317d9c2158a5b137b8a4402fcb8c68679c"} Jun 08 12:08:24.590809 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:24.590762 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:08:24.591289 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:24.590989 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:08:24.591289 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:24.591085 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:56.59106076 +0000 UTC m=+99.230902195 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:08:27.193721 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:27.193691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:27.267634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:27.267597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:08:29.121779 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:29.121687 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9j7b8" event={"ID":"6741b66e-5e5c-4be3-8f51-4b1093928dc0","Type":"ContainerStarted","Data":"49b8d5696ee0489d1d5757d10584b2349b6318503d768ab24f5fecc76ecf0f41"} Jun 08 12:08:29.121779 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:29.121775 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:08:29.123187 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:29.123166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-446pr" event={"ID":"80821cae-47d3-482a-8fdd-ac42bcf3e4f7","Type":"ContainerStarted","Data":"67e242325a6d35618f3ff122fc434235992953b401de5c269c75bee3b99d7c57"} Jun 08 12:08:29.137255 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:29.137213 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9j7b8" podStartSLOduration=66.287376274 podStartE2EDuration="1m11.137202087s" totalImagesPullingTime="4.849825813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:08:29.136743253 +0000 UTC m=+71.776584700" watchObservedRunningTime="2026-06-08 12:08:29.137202087 +0000 UTC m=+71.777043520" Jun 08 12:08:29.151130 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:29.151093 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-446pr" podStartSLOduration=66.275516025 podStartE2EDuration="1m11.151083418s" totalImagesPullingTime="4.875567393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:08:29.150833201 +0000 UTC m=+71.790674634" watchObservedRunningTime="2026-06-08 12:08:29.151083418 +0000 UTC m=+71.790924853" Jun 08 12:08:56.603112 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:08:56.602987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:08:56.603581 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:56.603134 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:08:56.603581 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:08:56.603194 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:10:00.603179658 +0000 UTC m=+163.243021069 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:09:00.130011 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:09:00.129967 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9j7b8" Jun 08 12:09:27.601286 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:09:27.601231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:09:27.601872 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:09:27.601407 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 12:09:27.601872 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:09:27.601504 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs podName:bfd9e9d8-9609-43d5-a45c-e58816b43118 nodeName:}" failed. No retries permitted until 2026-06-08 12:11:29.601482512 +0000 UTC m=+252.241323924 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs") pod "network-metrics-daemon-wr6xl" (UID: "bfd9e9d8-9609-43d5-a45c-e58816b43118") : secret "metrics-daemon-secret" not found Jun 08 12:09:40.373184 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:09:40.373148 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:09:55.705086 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:09:55.705021 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8gqn6" podUID="55247bf6-9e28-41a7-b925-b8e43914ef45" Jun 08 12:09:56.367759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:09:56.367731 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8gqn6" Jun 08 12:09:56.865420 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:09:56.865371 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-wr6xl" podUID="bfd9e9d8-9609-43d5-a45c-e58816b43118" Jun 08 12:10:00.622801 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:00.622762 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:10:00.623316 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:00.622879 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:10:00.623316 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:00.622931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls podName:55247bf6-9e28-41a7-b925-b8e43914ef45 nodeName:}" failed. No retries permitted until 2026-06-08 12:12:02.622917436 +0000 UTC m=+285.262758849 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls") pod "dns-default-8gqn6" (UID: "55247bf6-9e28-41a7-b925-b8e43914ef45") : secret "dns-default-metrics-tls" not found Jun 08 12:10:01.380799 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:01.380763 2580 generic.go:356] "Generic (PLEG): container finished" podID="fdecc9cb-450e-41a7-a3b1-4f656f24fed0" containerID="434ee0b034dda347a31fb02ac463af3f6756845838220eef49fe181a52e68285" exitCode=255 Jun 08 12:10:01.380981 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:01.380838 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" event={"ID":"fdecc9cb-450e-41a7-a3b1-4f656f24fed0","Type":"ContainerDied","Data":"434ee0b034dda347a31fb02ac463af3f6756845838220eef49fe181a52e68285"} Jun 08 12:10:01.386132 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:01.386114 2580 scope.go:117] "RemoveContainer" containerID="434ee0b034dda347a31fb02ac463af3f6756845838220eef49fe181a52e68285" Jun 08 12:10:02.384857 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:02.384823 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" event={"ID":"fdecc9cb-450e-41a7-a3b1-4f656f24fed0","Type":"ContainerStarted","Data":"e3b5dcf2c4f0dcbe9e4ed5ae0519269fbe60f92e24d9bc300a2fcc893ffb3742"} Jun 08 12:10:03.588404 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:03.588362 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:04.052592 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:04.052546 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" podUID="9ae6b08f-8528-497b-afbc-83f62e504b14" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused" Jun 08 12:10:04.391930 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:04.391851 2580 generic.go:356] "Generic (PLEG): container finished" podID="9ae6b08f-8528-497b-afbc-83f62e504b14" containerID="07c042bba791c5b67994a23a039f05a10da0d11f574325d7ee054fb1953f8e28" exitCode=1 Jun 08 12:10:04.391930 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:04.391887 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" event={"ID":"9ae6b08f-8528-497b-afbc-83f62e504b14","Type":"ContainerDied","Data":"07c042bba791c5b67994a23a039f05a10da0d11f574325d7ee054fb1953f8e28"} Jun 08 12:10:04.392192 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:04.392179 2580 scope.go:117] "RemoveContainer" containerID="07c042bba791c5b67994a23a039f05a10da0d11f574325d7ee054fb1953f8e28" Jun 08 12:10:05.396578 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:05.396539 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" event={"ID":"9ae6b08f-8528-497b-afbc-83f62e504b14","Type":"ContainerStarted","Data":"d3f3dc796b3fe076efc5551dde13cb053bce254e750c1428d870d03549620a4f"} Jun 08 12:10:05.396969 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:05.396821 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:10:05.397438 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:05.397423 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:10:07.855686 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:07.855603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:10:39.158831 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.158790 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-kvr8v"] Jun 08 12:10:39.161537 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.161522 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b68d8d847-bg5jw"] Jun 08 12:10:39.161651 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.161632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.164261 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.164239 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.164785 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.164766 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 08 12:10:39.164937 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.164918 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-88srl\"" Jun 08 12:10:39.165035 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.164984 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 08 12:10:39.165101 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.165034 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 08 12:10:39.165101 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.165085 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 08 12:10:39.169321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.169301 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 12:10:39.169439 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.169321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 12:10:39.170466 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.170442 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 08 12:10:39.170563 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.170502 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jwnkf\"" Jun 08 12:10:39.170628 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.170444 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 12:10:39.174824 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.174803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-kvr8v"] Jun 08 12:10:39.175041 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.175024 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 12:10:39.181073 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.181051 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b68d8d847-bg5jw"] Jun 08 12:10:39.183239 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-trusted-ca\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.183332 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-installation-pull-secrets\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.183332 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhw4c\" (UniqueName: \"kubernetes.io/projected/cacc1679-4496-434a-b777-29c9f516ac87-kube-api-access-vhw4c\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.183417 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-bound-sa-token\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.183417 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cacc1679-4496-434a-b777-29c9f516ac87-config\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.183499 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cacc1679-4496-434a-b777-29c9f516ac87-serving-cert\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.183499 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cacc1679-4496-434a-b777-29c9f516ac87-trusted-ca\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.183499 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-image-registry-private-configuration\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.183624 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183508 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-certificates\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.183624 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.183624 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49pb6\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-kube-api-access-49pb6\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.183762 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.183646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e12cb94-4c4e-405b-876a-8cab95b9c907-ca-trust-extracted\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284131 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284131 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49pb6\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-kube-api-access-49pb6\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e12cb94-4c4e-405b-876a-8cab95b9c907-ca-trust-extracted\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-trusted-ca\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-installation-pull-secrets\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:39.284266 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhw4c\" (UniqueName: \"kubernetes.io/projected/cacc1679-4496-434a-b777-29c9f516ac87-kube-api-access-vhw4c\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:39.284282 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b68d8d847-bg5jw: secret "image-registry-tls" not found Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-bound-sa-token\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cacc1679-4496-434a-b777-29c9f516ac87-config\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:39.284351 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls podName:9e12cb94-4c4e-405b-876a-8cab95b9c907 nodeName:}" failed. No retries permitted until 2026-06-08 12:10:39.78432944 +0000 UTC m=+202.424170871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls") pod "image-registry-7b68d8d847-bg5jw" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907") : secret "image-registry-tls" not found Jun 08 12:10:39.284392 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cacc1679-4496-434a-b777-29c9f516ac87-serving-cert\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.284879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cacc1679-4496-434a-b777-29c9f516ac87-trusted-ca\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.284879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-image-registry-private-configuration\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-certificates\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.284879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.284668 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e12cb94-4c4e-405b-876a-8cab95b9c907-ca-trust-extracted\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.285110 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.285020 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-certificates\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.285168 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.285119 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cacc1679-4496-434a-b777-29c9f516ac87-config\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.285478 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.285457 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-trusted-ca\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.285587 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.285562 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cacc1679-4496-434a-b777-29c9f516ac87-trusted-ca\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.286923 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.286896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cacc1679-4496-434a-b777-29c9f516ac87-serving-cert\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.286923 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.286900 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-installation-pull-secrets\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.287178 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.287155 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-image-registry-private-configuration\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.292901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.292881 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-bound-sa-token\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.293004 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.292961 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhw4c\" (UniqueName: \"kubernetes.io/projected/cacc1679-4496-434a-b777-29c9f516ac87-kube-api-access-vhw4c\") pod \"console-operator-668fdbf85c-kvr8v\" (UID: \"cacc1679-4496-434a-b777-29c9f516ac87\") " pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.293225 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.293206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49pb6\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-kube-api-access-49pb6\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.363211 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.363181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:39.418087 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.418025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:39.472069 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.472041 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:39.592962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.592920 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-kvr8v"] Jun 08 12:10:39.596173 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:10:39.596146 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcacc1679_4496_434a_b777_29c9f516ac87.slice/crio-032bd5b8a35041e3460c808c29ceca13faab8b176e6285629710d0464d989db5 WatchSource:0}: Error finding container 032bd5b8a35041e3460c808c29ceca13faab8b176e6285629710d0464d989db5: Status 404 returned error can't find the container with id 032bd5b8a35041e3460c808c29ceca13faab8b176e6285629710d0464d989db5 Jun 08 12:10:39.788419 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:39.788377 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:39.788609 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:39.788493 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:10:39.788609 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:39.788506 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b68d8d847-bg5jw: secret "image-registry-tls" not found Jun 08 12:10:39.788609 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:39.788559 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls podName:9e12cb94-4c4e-405b-876a-8cab95b9c907 nodeName:}" failed. No retries permitted until 2026-06-08 12:10:40.788542686 +0000 UTC m=+203.428384098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls") pod "image-registry-7b68d8d847-bg5jw" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907") : secret "image-registry-tls" not found Jun 08 12:10:40.496438 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:40.496402 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" event={"ID":"cacc1679-4496-434a-b777-29c9f516ac87","Type":"ContainerStarted","Data":"032bd5b8a35041e3460c808c29ceca13faab8b176e6285629710d0464d989db5"} Jun 08 12:10:40.794845 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:40.794758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:40.795019 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:40.794881 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:10:40.795019 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:40.794894 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b68d8d847-bg5jw: secret "image-registry-tls" not found Jun 08 12:10:40.795019 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:40.794970 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls podName:9e12cb94-4c4e-405b-876a-8cab95b9c907 nodeName:}" failed. No retries permitted until 2026-06-08 12:10:42.794930835 +0000 UTC m=+205.434772270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls") pod "image-registry-7b68d8d847-bg5jw" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907") : secret "image-registry-tls" not found Jun 08 12:10:41.574214 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:41.574178 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:42.504257 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:42.504229 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/0.log" Jun 08 12:10:42.504448 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:42.504269 2580 generic.go:356] "Generic (PLEG): container finished" podID="cacc1679-4496-434a-b777-29c9f516ac87" containerID="9a56bdd78aba5e0182fde1dad7f9cc4cfb0cf267ef96a7c3b423e841b2b4af55" exitCode=255 Jun 08 12:10:42.504448 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:42.504301 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" event={"ID":"cacc1679-4496-434a-b777-29c9f516ac87","Type":"ContainerDied","Data":"9a56bdd78aba5e0182fde1dad7f9cc4cfb0cf267ef96a7c3b423e841b2b4af55"} Jun 08 12:10:42.504617 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:42.504602 2580 scope.go:117] "RemoveContainer" containerID="9a56bdd78aba5e0182fde1dad7f9cc4cfb0cf267ef96a7c3b423e841b2b4af55" Jun 08 12:10:42.737623 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:42.737586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:42.808175 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:42.808114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:42.808263 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:42.808255 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:10:42.808301 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:42.808267 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b68d8d847-bg5jw: secret "image-registry-tls" not found Jun 08 12:10:42.808334 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:42.808327 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls podName:9e12cb94-4c4e-405b-876a-8cab95b9c907 nodeName:}" failed. No retries permitted until 2026-06-08 12:10:46.80830985 +0000 UTC m=+209.448151272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls") pod "image-registry-7b68d8d847-bg5jw" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907") : secret "image-registry-tls" not found Jun 08 12:10:42.837821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:42.837795 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:42.948522 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:42.948494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:43.508146 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:43.508117 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/1.log" Jun 08 12:10:43.508547 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:43.508526 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/0.log" Jun 08 12:10:43.508698 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:43.508563 2580 generic.go:356] "Generic (PLEG): container finished" podID="cacc1679-4496-434a-b777-29c9f516ac87" containerID="a1f78bebd5d7cae73dbc7794755c6f7d7e4c12621eb989b91cd476c4c4ef45d0" exitCode=255 Jun 08 12:10:43.508698 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:43.508596 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" event={"ID":"cacc1679-4496-434a-b777-29c9f516ac87","Type":"ContainerDied","Data":"a1f78bebd5d7cae73dbc7794755c6f7d7e4c12621eb989b91cd476c4c4ef45d0"} Jun 08 12:10:43.508698 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:43.508646 2580 scope.go:117] "RemoveContainer" containerID="9a56bdd78aba5e0182fde1dad7f9cc4cfb0cf267ef96a7c3b423e841b2b4af55" Jun 08 12:10:43.509056 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:43.509029 2580 scope.go:117] "RemoveContainer" containerID="a1f78bebd5d7cae73dbc7794755c6f7d7e4c12621eb989b91cd476c4c4ef45d0" Jun 08 12:10:43.509198 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:43.509179 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-kvr8v_openshift-console-operator(cacc1679-4496-434a-b777-29c9f516ac87)\"" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" podUID="cacc1679-4496-434a-b777-29c9f516ac87" Jun 08 12:10:43.797665 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:43.797590 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:43.858938 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:43.858907 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:44.238597 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:44.238569 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:44.338541 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:44.338505 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:44.438966 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:44.438902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:44.501460 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:44.501396 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:44.512795 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:44.512775 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/1.log" Jun 08 12:10:44.513104 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:44.513089 2580 scope.go:117] "RemoveContainer" containerID="a1f78bebd5d7cae73dbc7794755c6f7d7e4c12621eb989b91cd476c4c4ef45d0" Jun 08 12:10:44.513243 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:44.513225 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-kvr8v_openshift-console-operator(cacc1679-4496-434a-b777-29c9f516ac87)\"" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" podUID="cacc1679-4496-434a-b777-29c9f516ac87" Jun 08 12:10:44.938103 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:44.938021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:45.039751 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:45.039720 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:46.577230 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:46.577197 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:46.630461 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:46.630435 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:46.839364 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:46.839271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:46.839507 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:46.839373 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:10:46.839507 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:46.839392 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b68d8d847-bg5jw: secret "image-registry-tls" not found Jun 08 12:10:46.839507 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:46.839443 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls podName:9e12cb94-4c4e-405b-876a-8cab95b9c907 nodeName:}" failed. No retries permitted until 2026-06-08 12:10:54.839428978 +0000 UTC m=+217.479270389 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls") pod "image-registry-7b68d8d847-bg5jw" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907") : secret "image-registry-tls" not found Jun 08 12:10:47.399742 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:47.399715 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w469r_e2b2638a-0ff1-4b0f-9d68-10d8278f33a2/dns-node-resolver/0.log" Jun 08 12:10:47.581869 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:47.581837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:48.192596 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:48.192567 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:48.401716 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:48.401685 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fhmb9_66348e6c-fe0c-43c0-989c-fbd2c69373a1/node-ca/0.log" Jun 08 12:10:48.440001 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:48.439975 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:48.561675 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:48.561645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:48.643938 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:48.643905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:48.723504 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:48.723477 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:48.803977 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:48.803954 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5rmvq_636ec464-e603-4917-a35e-cdfa55a4491b/serve-healthcheck-canary/0.log" Jun 08 12:10:48.855867 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:48.855785 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:49.472975 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:49.472937 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:49.473137 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:49.473022 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:10:49.473305 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:49.473293 2580 scope.go:117] "RemoveContainer" containerID="a1f78bebd5d7cae73dbc7794755c6f7d7e4c12621eb989b91cd476c4c4ef45d0" Jun 08 12:10:49.473447 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:10:49.473430 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-kvr8v_openshift-console-operator(cacc1679-4496-434a-b777-29c9f516ac87)\"" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" podUID="cacc1679-4496-434a-b777-29c9f516ac87" Jun 08 12:10:49.843343 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:49.843307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:50.249465 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:50.249430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:50.448638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:50.448611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:50.642461 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:50.642387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:50.851975 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:50.851935 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:51.412606 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:51.412570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:52.001163 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:52.001128 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:52.101199 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:52.101173 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:52.249406 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:52.249376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:52.450139 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:52.450064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:52.636176 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:52.636147 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:52.852901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:52.852873 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:54.046385 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:54.046345 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:54.246080 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:54.246045 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:54.483829 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:54.483805 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:54.545034 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:54.545006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:54.639874 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:54.639846 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:54.834053 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:54.833975 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:54.893851 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:54.893815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:54.896244 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:54.896214 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"image-registry-7b68d8d847-bg5jw\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:55.077015 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:55.076967 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:55.206317 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:55.206282 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b68d8d847-bg5jw"] Jun 08 12:10:55.209256 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:10:55.209222 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e12cb94_4c4e_405b_876a_8cab95b9c907.slice/crio-e99fad0485041349432d5ea2a1349ce09fa88386b3e3cfac081b89b10d808ae3 WatchSource:0}: Error finding container e99fad0485041349432d5ea2a1349ce09fa88386b3e3cfac081b89b10d808ae3: Status 404 returned error can't find the container with id e99fad0485041349432d5ea2a1349ce09fa88386b3e3cfac081b89b10d808ae3 Jun 08 12:10:55.544641 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:55.544611 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" event={"ID":"9e12cb94-4c4e-405b-876a-8cab95b9c907","Type":"ContainerStarted","Data":"d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b"} Jun 08 12:10:55.544805 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:55.544660 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" event={"ID":"9e12cb94-4c4e-405b-876a-8cab95b9c907","Type":"ContainerStarted","Data":"e99fad0485041349432d5ea2a1349ce09fa88386b3e3cfac081b89b10d808ae3"} Jun 08 12:10:55.544805 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:55.544736 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:10:55.565311 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:55.565255 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" podStartSLOduration=16.565235918 podStartE2EDuration="16.565235918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:10:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:10:55.56430677 +0000 UTC m=+218.204148203" watchObservedRunningTime="2026-06-08 12:10:55.565235918 +0000 UTC m=+218.205077353" Jun 08 12:10:55.792310 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:55.792274 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:56.038803 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:56.038771 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:56.237100 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:56.237067 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:56.439504 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:56.439427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:56.499695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:56.499666 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:56.640472 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:56.640434 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:56.836098 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:56.836065 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:57.637579 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:57.637546 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:57.838996 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:57.838966 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:58.244771 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:58.244740 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:58.311358 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:58.311295 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:58.643137 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:58.643054 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:10:58.847716 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:10:58.847684 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:04.855145 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:04.855109 2580 scope.go:117] "RemoveContainer" containerID="a1f78bebd5d7cae73dbc7794755c6f7d7e4c12621eb989b91cd476c4c4ef45d0" Jun 08 12:11:05.573968 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:05.573912 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:11:05.574372 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:05.574355 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/1.log" Jun 08 12:11:05.574432 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:05.574391 2580 generic.go:356] "Generic (PLEG): container finished" podID="cacc1679-4496-434a-b777-29c9f516ac87" containerID="2b52ab0746a01549750fbdc0eb52266791982f781bde1e6bff0a8fdf76b73e98" exitCode=255 Jun 08 12:11:05.574471 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:05.574453 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" event={"ID":"cacc1679-4496-434a-b777-29c9f516ac87","Type":"ContainerDied","Data":"2b52ab0746a01549750fbdc0eb52266791982f781bde1e6bff0a8fdf76b73e98"} Jun 08 12:11:05.574519 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:05.574507 2580 scope.go:117] "RemoveContainer" containerID="a1f78bebd5d7cae73dbc7794755c6f7d7e4c12621eb989b91cd476c4c4ef45d0" Jun 08 12:11:05.574810 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:05.574790 2580 scope.go:117] "RemoveContainer" containerID="2b52ab0746a01549750fbdc0eb52266791982f781bde1e6bff0a8fdf76b73e98" Jun 08 12:11:05.575006 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:11:05.574988 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-kvr8v_openshift-console-operator(cacc1679-4496-434a-b777-29c9f516ac87)\"" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" podUID="cacc1679-4496-434a-b777-29c9f516ac87" Jun 08 12:11:06.578574 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:06.578548 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:11:09.472219 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.472185 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:11:09.472616 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.472244 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:11:09.472616 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.472510 2580 scope.go:117] "RemoveContainer" containerID="2b52ab0746a01549750fbdc0eb52266791982f781bde1e6bff0a8fdf76b73e98" Jun 08 12:11:09.472687 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:11:09.472650 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-kvr8v_openshift-console-operator(cacc1679-4496-434a-b777-29c9f516ac87)\"" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" podUID="cacc1679-4496-434a-b777-29c9f516ac87" Jun 08 12:11:09.604856 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.604825 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b68d8d847-bg5jw"] Jun 08 12:11:09.625670 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.625642 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wjt65"] Jun 08 12:11:09.629151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.629132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.631758 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.631735 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 08 12:11:09.631953 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.631928 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5b5xn\"" Jun 08 12:11:09.632005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.631958 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 08 12:11:09.632083 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.632067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 08 12:11:09.632135 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.632122 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 08 12:11:09.647169 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.647150 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wjt65"] Jun 08 12:11:09.675574 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.675550 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:09.702675 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.702651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17f21fbe-654b-40d5-a3b4-5be67414fa01-data-volume\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.702759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.702689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjrv6\" (UniqueName: \"kubernetes.io/projected/17f21fbe-654b-40d5-a3b4-5be67414fa01-kube-api-access-rjrv6\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.702759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.702710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17f21fbe-654b-40d5-a3b4-5be67414fa01-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.702759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.702743 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17f21fbe-654b-40d5-a3b4-5be67414fa01-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.702868 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.702783 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17f21fbe-654b-40d5-a3b4-5be67414fa01-crio-socket\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.775652 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.775629 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:09.803853 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.803827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrv6\" (UniqueName: \"kubernetes.io/projected/17f21fbe-654b-40d5-a3b4-5be67414fa01-kube-api-access-rjrv6\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.803936 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.803860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17f21fbe-654b-40d5-a3b4-5be67414fa01-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.803936 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.803879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17f21fbe-654b-40d5-a3b4-5be67414fa01-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.804057 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.803934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17f21fbe-654b-40d5-a3b4-5be67414fa01-crio-socket\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.804057 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.803996 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17f21fbe-654b-40d5-a3b4-5be67414fa01-data-volume\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.804157 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.804067 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17f21fbe-654b-40d5-a3b4-5be67414fa01-crio-socket\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.804369 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.804350 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17f21fbe-654b-40d5-a3b4-5be67414fa01-data-volume\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.804537 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.804522 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17f21fbe-654b-40d5-a3b4-5be67414fa01-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.806165 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.806149 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17f21fbe-654b-40d5-a3b4-5be67414fa01-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.814271 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.814253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjrv6\" (UniqueName: \"kubernetes.io/projected/17f21fbe-654b-40d5-a3b4-5be67414fa01-kube-api-access-rjrv6\") pod \"insights-runtime-extractor-wjt65\" (UID: \"17f21fbe-654b-40d5-a3b4-5be67414fa01\") " pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:09.937424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:09.937394 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wjt65" Jun 08 12:11:10.063777 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:10.063679 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wjt65"] Jun 08 12:11:10.066755 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:11:10.066725 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f21fbe_654b_40d5_a3b4_5be67414fa01.slice/crio-2e9797e3fc6c77ae360e0c10712e4502073916c625175f65e62ff90e28430aa6 WatchSource:0}: Error finding container 2e9797e3fc6c77ae360e0c10712e4502073916c625175f65e62ff90e28430aa6: Status 404 returned error can't find the container with id 2e9797e3fc6c77ae360e0c10712e4502073916c625175f65e62ff90e28430aa6 Jun 08 12:11:10.591888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:10.591856 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wjt65" event={"ID":"17f21fbe-654b-40d5-a3b4-5be67414fa01","Type":"ContainerStarted","Data":"8edcefd6622cfdef6712edf45c6096674416305ec018d050059369b416c8b6a4"} Jun 08 12:11:10.592269 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:10.591913 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wjt65" event={"ID":"17f21fbe-654b-40d5-a3b4-5be67414fa01","Type":"ContainerStarted","Data":"2e9797e3fc6c77ae360e0c10712e4502073916c625175f65e62ff90e28430aa6"} Jun 08 12:11:11.597073 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:11.597025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wjt65" event={"ID":"17f21fbe-654b-40d5-a3b4-5be67414fa01","Type":"ContainerStarted","Data":"03bc36f2d434074c42867bc06418e512f5487b53581fb923aa951a3fa9128a00"} Jun 08 12:11:12.601535 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:12.601503 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wjt65" event={"ID":"17f21fbe-654b-40d5-a3b4-5be67414fa01","Type":"ContainerStarted","Data":"fe1a4e996f86454a2b8b682ce61f94a3e479988d29b9a8261e17b67231ec6397"} Jun 08 12:11:12.622568 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:12.622516 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wjt65" podStartSLOduration=1.3322202 podStartE2EDuration="3.622502474s" totalImagesPullingTime="2.290282274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:11:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:12.62185406 +0000 UTC m=+235.261695494" watchObservedRunningTime="2026-06-08 12:11:12.622502474 +0000 UTC m=+235.262343908" Jun 08 12:11:14.171989 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:14.171934 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:14.272103 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:14.272071 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:14.374268 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:14.374236 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:14.544295 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:14.544267 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:14.746281 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:14.746251 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:14.847451 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:14.847364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:14.899912 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:14.899887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:16.361210 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.361175 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:16.400036 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.400008 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx"] Jun 08 12:11:16.403093 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.403078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.405877 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.405855 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 08 12:11:16.407067 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.407048 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 08 12:11:16.407067 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.407057 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 08 12:11:16.407229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.407175 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 08 12:11:16.407229 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.407182 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r6lrz\"" Jun 08 12:11:16.407337 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.407287 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 08 12:11:16.413151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.413128 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx"] Jun 08 12:11:16.433021 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.432989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:16.535644 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.535618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:16.554226 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.554201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55srz\" (UniqueName: \"kubernetes.io/projected/8dfed933-9024-45d1-99e9-42bb7381cd43-kube-api-access-55srz\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.554356 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.554239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dfed933-9024-45d1-99e9-42bb7381cd43-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.554356 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.554332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8dfed933-9024-45d1-99e9-42bb7381cd43-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.554464 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.554381 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8dfed933-9024-45d1-99e9-42bb7381cd43-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.655428 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.655359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55srz\" (UniqueName: \"kubernetes.io/projected/8dfed933-9024-45d1-99e9-42bb7381cd43-kube-api-access-55srz\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.655428 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.655394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dfed933-9024-45d1-99e9-42bb7381cd43-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.655577 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.655431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8dfed933-9024-45d1-99e9-42bb7381cd43-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.655577 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.655463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8dfed933-9024-45d1-99e9-42bb7381cd43-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.656179 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.656158 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dfed933-9024-45d1-99e9-42bb7381cd43-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.657810 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.657786 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8dfed933-9024-45d1-99e9-42bb7381cd43-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.657891 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.657789 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8dfed933-9024-45d1-99e9-42bb7381cd43-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.664304 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.664280 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55srz\" (UniqueName: \"kubernetes.io/projected/8dfed933-9024-45d1-99e9-42bb7381cd43-kube-api-access-55srz\") pod \"prometheus-operator-7f7d445d84-ltnxx\" (UID: \"8dfed933-9024-45d1-99e9-42bb7381cd43\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.712225 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.712199 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" Jun 08 12:11:16.833983 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:16.833933 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx"] Jun 08 12:11:16.836715 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:11:16.836687 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfed933_9024_45d1_99e9_42bb7381cd43.slice/crio-625660b28322356a192dc1687650b8f98944b9c66c14e0f794bb2664f3f48d28 WatchSource:0}: Error finding container 625660b28322356a192dc1687650b8f98944b9c66c14e0f794bb2664f3f48d28: Status 404 returned error can't find the container with id 625660b28322356a192dc1687650b8f98944b9c66c14e0f794bb2664f3f48d28 Jun 08 12:11:17.615960 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:17.615912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" event={"ID":"8dfed933-9024-45d1-99e9-42bb7381cd43","Type":"ContainerStarted","Data":"625660b28322356a192dc1687650b8f98944b9c66c14e0f794bb2664f3f48d28"} Jun 08 12:11:18.323629 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:18.323549 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:18.620803 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:18.620712 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" event={"ID":"8dfed933-9024-45d1-99e9-42bb7381cd43","Type":"ContainerStarted","Data":"00f4513ce82ba44315ba7e17eaeb97804c237eeb977fd3a9d3e37e8d7257e6de"} Jun 08 12:11:18.620803 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:18.620766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" event={"ID":"8dfed933-9024-45d1-99e9-42bb7381cd43","Type":"ContainerStarted","Data":"8ad9f8f4c729266664ec33cdace07a2f5cee0825b910829241f48f19efc279ad"} Jun 08 12:11:18.636478 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:18.636432 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ltnxx" podStartSLOduration=1.4110109259999999 podStartE2EDuration="2.636419066s" totalImagesPullingTime="1.22540814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:18.636257471 +0000 UTC m=+241.276098905" watchObservedRunningTime="2026-06-08 12:11:18.636419066 +0000 UTC m=+241.276260500" Jun 08 12:11:19.610246 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:19.610212 2580 patch_prober.go:28] interesting pod/image-registry-7b68d8d847-bg5jw container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 12:11:19.610413 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:19.610262 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" podUID="9e12cb94-4c4e-405b-876a-8cab95b9c907" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 12:11:20.499027 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.498992 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:20.592028 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.591999 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:20.712191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.712159 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:20.734591 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.734564 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l"] Jun 08 12:11:20.737661 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.737646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.740408 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.740387 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 08 12:11:20.740517 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.740430 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 08 12:11:20.740753 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.740739 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f9vwr\"" Jun 08 12:11:20.749888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.749838 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l"] Jun 08 12:11:20.772430 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.772402 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xmw5n"] Jun 08 12:11:20.775527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.775511 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.778030 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.778000 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 08 12:11:20.778149 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.778121 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p49nf\"" Jun 08 12:11:20.778149 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.778132 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 08 12:11:20.778281 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.778197 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 08 12:11:20.807785 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.807762 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:20.855301 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.855275 2580 scope.go:117] "RemoveContainer" containerID="2b52ab0746a01549750fbdc0eb52266791982f781bde1e6bff0a8fdf76b73e98" Jun 08 12:11:20.855453 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:11:20.855434 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-kvr8v_openshift-console-operator(cacc1679-4496-434a-b777-29c9f516ac87)\"" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" podUID="cacc1679-4496-434a-b777-29c9f516ac87" Jun 08 12:11:20.872334 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.872310 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:20.887434 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5jfv\" (UniqueName: \"kubernetes.io/projected/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-kube-api-access-f5jfv\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.887559 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9n2p\" (UniqueName: \"kubernetes.io/projected/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-kube-api-access-h9n2p\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.887559 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.887559 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-metrics-client-ca\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.887723 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.887723 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887623 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-textfile\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.887723 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887692 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-tls\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.887821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.887821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-sys\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.887821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-root\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.887961 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.887961 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887837 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.887961 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.887861 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-wtmp\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.976206 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.976176 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:20.988238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9n2p\" (UniqueName: \"kubernetes.io/projected/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-kube-api-access-h9n2p\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988331 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.988331 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-metrics-client-ca\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.988416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-textfile\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-tls\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988598 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-sys\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988598 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-root\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988598 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.988598 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988598 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-sys\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988844 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988642 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-wtmp\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.988844 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988684 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5jfv\" (UniqueName: \"kubernetes.io/projected/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-kube-api-access-f5jfv\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.988975 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.988850 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-wtmp\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.989056 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.989030 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-metrics-client-ca\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.989114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.989032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-textfile\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.989114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.989099 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-root\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.989727 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.989706 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.990020 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.990000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.991274 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.991218 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.991437 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.991416 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-node-exporter-tls\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.991514 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.991416 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.991610 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.991590 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:20.995670 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.995644 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9n2p\" (UniqueName: \"kubernetes.io/projected/0c1606b4-24e2-49a3-bc12-47e549a6c8d4-kube-api-access-h9n2p\") pod \"node-exporter-xmw5n\" (UID: \"0c1606b4-24e2-49a3-bc12-47e549a6c8d4\") " pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:20.996312 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:20.996296 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5jfv\" (UniqueName: \"kubernetes.io/projected/1bc28b4b-9fe1-43f3-b9c8-2154200215bd-kube-api-access-f5jfv\") pod \"openshift-state-metrics-65f78d5c66-lrl7l\" (UID: \"1bc28b4b-9fe1-43f3-b9c8-2154200215bd\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:21.046367 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.046295 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" Jun 08 12:11:21.074307 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.074278 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:21.085420 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.085396 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xmw5n" Jun 08 12:11:21.097333 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:11:21.097300 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1606b4_24e2_49a3_bc12_47e549a6c8d4.slice/crio-789a44ded8d539773b33fd47e13abc284ad5bc87b0b1b8aba1bd9c442936028c WatchSource:0}: Error finding container 789a44ded8d539773b33fd47e13abc284ad5bc87b0b1b8aba1bd9c442936028c: Status 404 returned error can't find the container with id 789a44ded8d539773b33fd47e13abc284ad5bc87b0b1b8aba1bd9c442936028c Jun 08 12:11:21.177691 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.177661 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l"] Jun 08 12:11:21.178230 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.178198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:21.181653 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:11:21.181621 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc28b4b_9fe1_43f3_b9c8_2154200215bd.slice/crio-aded63da66636a850747c88c2f92fc0ee634c135d864648638b13b684c78da4d WatchSource:0}: Error finding container aded63da66636a850747c88c2f92fc0ee634c135d864648638b13b684c78da4d: Status 404 returned error can't find the container with id aded63da66636a850747c88c2f92fc0ee634c135d864648638b13b684c78da4d Jun 08 12:11:21.633102 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.633018 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xmw5n" event={"ID":"0c1606b4-24e2-49a3-bc12-47e549a6c8d4","Type":"ContainerStarted","Data":"789a44ded8d539773b33fd47e13abc284ad5bc87b0b1b8aba1bd9c442936028c"} Jun 08 12:11:21.634729 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.634707 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" event={"ID":"1bc28b4b-9fe1-43f3-b9c8-2154200215bd","Type":"ContainerStarted","Data":"8ab076d3385aabdb0e53a9c2b6a39acd460229cdde305b41c5e05c2f08a90985"} Jun 08 12:11:21.634840 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.634750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" event={"ID":"1bc28b4b-9fe1-43f3-b9c8-2154200215bd","Type":"ContainerStarted","Data":"4a582d0d3d5d39812ba34bc9aaf52673ea0cb611703907ad9c382401bb5efd7a"} Jun 08 12:11:21.634840 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.634769 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" event={"ID":"1bc28b4b-9fe1-43f3-b9c8-2154200215bd","Type":"ContainerStarted","Data":"aded63da66636a850747c88c2f92fc0ee634c135d864648638b13b684c78da4d"} Jun 08 12:11:21.645526 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.645502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:21.746724 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.746692 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:21.826401 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.826368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:21.990146 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:21.990101 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:22.640456 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.640369 2580 generic.go:356] "Generic (PLEG): container finished" podID="0c1606b4-24e2-49a3-bc12-47e549a6c8d4" containerID="a38232530604c3a1ae50d5b2c85e828260c7f9163f489301dbc282dd3bcc8abe" exitCode=0 Jun 08 12:11:22.640456 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.640417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xmw5n" event={"ID":"0c1606b4-24e2-49a3-bc12-47e549a6c8d4","Type":"ContainerDied","Data":"a38232530604c3a1ae50d5b2c85e828260c7f9163f489301dbc282dd3bcc8abe"} Jun 08 12:11:22.843899 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.843864 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7486f69c5-24lzt"] Jun 08 12:11:22.847545 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.847517 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:22.851161 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.851133 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 08 12:11:22.851291 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.851271 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 08 12:11:22.852120 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.852087 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3f2ki5vpgciou\"" Jun 08 12:11:22.852339 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.852302 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 08 12:11:22.852339 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.852331 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 08 12:11:22.852498 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.852380 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 08 12:11:22.852603 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.852591 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kvds8\"" Jun 08 12:11:22.860790 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:22.860765 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7486f69c5-24lzt"] Jun 08 12:11:23.005938 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.005913 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3016476f-e848-49db-89be-0032c86f0c3a-metrics-client-ca\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.006082 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.005967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.006082 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.005995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.006082 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.006039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdvgc\" (UniqueName: \"kubernetes.io/projected/3016476f-e848-49db-89be-0032c86f0c3a-kube-api-access-cdvgc\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.006082 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.006059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-tls\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.006235 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.006098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-grpc-tls\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.006235 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.006150 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.006235 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.006188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.107166 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.107128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3016476f-e848-49db-89be-0032c86f0c3a-metrics-client-ca\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.107166 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.107164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.107428 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.107189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.107428 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.107213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdvgc\" (UniqueName: \"kubernetes.io/projected/3016476f-e848-49db-89be-0032c86f0c3a-kube-api-access-cdvgc\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.107428 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.107232 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-tls\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.107428 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.107273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-grpc-tls\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.107428 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.107296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.107428 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.107312 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.108111 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.108057 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3016476f-e848-49db-89be-0032c86f0c3a-metrics-client-ca\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.110025 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.109997 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.110162 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.110141 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-grpc-tls\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.110315 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.110287 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-tls\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.110377 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.110355 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.110423 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.110408 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.110460 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.110445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3016476f-e848-49db-89be-0032c86f0c3a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.116317 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.116300 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdvgc\" (UniqueName: \"kubernetes.io/projected/3016476f-e848-49db-89be-0032c86f0c3a-kube-api-access-cdvgc\") pod \"thanos-querier-7486f69c5-24lzt\" (UID: \"3016476f-e848-49db-89be-0032c86f0c3a\") " pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.158592 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.158506 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:23.296385 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.296357 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7486f69c5-24lzt"] Jun 08 12:11:23.298082 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:11:23.298054 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3016476f_e848_49db_89be_0032c86f0c3a.slice/crio-65f6982383be03dbf075758ee5421efc91576f6503109f61f7247c5531cd1efd WatchSource:0}: Error finding container 65f6982383be03dbf075758ee5421efc91576f6503109f61f7247c5531cd1efd: Status 404 returned error can't find the container with id 65f6982383be03dbf075758ee5421efc91576f6503109f61f7247c5531cd1efd Jun 08 12:11:23.646162 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.646125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xmw5n" event={"ID":"0c1606b4-24e2-49a3-bc12-47e549a6c8d4","Type":"ContainerStarted","Data":"7525785610d57fd39157756f056c009ab099ea74336b3e56b44fc8cc6d9628a3"} Jun 08 12:11:23.646611 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.646188 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xmw5n" event={"ID":"0c1606b4-24e2-49a3-bc12-47e549a6c8d4","Type":"ContainerStarted","Data":"c8b8448d75f1c48da2ef9f980a0637c6db4314f86b723c742997133738386e3c"} Jun 08 12:11:23.647290 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.647271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" event={"ID":"3016476f-e848-49db-89be-0032c86f0c3a","Type":"ContainerStarted","Data":"65f6982383be03dbf075758ee5421efc91576f6503109f61f7247c5531cd1efd"} Jun 08 12:11:23.649157 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.649134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" event={"ID":"1bc28b4b-9fe1-43f3-b9c8-2154200215bd","Type":"ContainerStarted","Data":"25a5457cb967fc1b0e3270bee0bd57d61dc9b33aab9d8b6756eb8dbabd7a343c"} Jun 08 12:11:23.666644 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.666597 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xmw5n" podStartSLOduration=2.536806953 podStartE2EDuration="3.66658184s" totalImagesPullingTime="1.065350648s" totalInitContainerRuntime="64.424239ms" isStatefulPod=true podCreationTimestamp="2026-06-08 12:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:23.665294981 +0000 UTC m=+246.305136416" watchObservedRunningTime="2026-06-08 12:11:23.66658184 +0000 UTC m=+246.306423275" Jun 08 12:11:23.684896 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.684841 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lrl7l" podStartSLOduration=2.051239868 podStartE2EDuration="3.684826003s" totalImagesPullingTime="1.633586135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:23.683986832 +0000 UTC m=+246.323828264" watchObservedRunningTime="2026-06-08 12:11:23.684826003 +0000 UTC m=+246.324667437" Jun 08 12:11:23.704055 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.704024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:23.805273 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.805220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:23.905286 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:23.905198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:24.229555 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:24.229517 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:25.525090 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.525057 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:25.528295 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.528268 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb"] Jun 08 12:11:25.532566 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.532542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:25.535097 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.535077 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 08 12:11:25.535191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.535099 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4dksw\"" Jun 08 12:11:25.542267 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.542202 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb"] Jun 08 12:11:25.628289 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.628250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7n4r\" (UniqueName: \"kubernetes.io/projected/2739c907-b011-4813-b88a-db69ed5688b1-kube-api-access-x7n4r\") pod \"monitoring-plugin-76b68cc874-dnrrb\" (UID: \"2739c907-b011-4813-b88a-db69ed5688b1\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:25.628439 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.628327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2739c907-b011-4813-b88a-db69ed5688b1-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-dnrrb\" (UID: \"2739c907-b011-4813-b88a-db69ed5688b1\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:25.658451 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.658369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" event={"ID":"3016476f-e848-49db-89be-0032c86f0c3a","Type":"ContainerStarted","Data":"70748575a84ea42f06cc077abe6b29e89bd3e96e570d5480fbea15d2eae71242"} Jun 08 12:11:25.658451 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.658435 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" event={"ID":"3016476f-e848-49db-89be-0032c86f0c3a","Type":"ContainerStarted","Data":"f98467a046e90099699fd2aa65017c7cc7c21f613cfd08c4e4c22106e1b9ad98"} Jun 08 12:11:25.658602 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.658462 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" event={"ID":"3016476f-e848-49db-89be-0032c86f0c3a","Type":"ContainerStarted","Data":"0313e8925bae15f89b33f9247c30aed72df9a9607a8d655d42e2306714511003"} Jun 08 12:11:25.724053 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.724020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:25.729167 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.729144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7n4r\" (UniqueName: \"kubernetes.io/projected/2739c907-b011-4813-b88a-db69ed5688b1-kube-api-access-x7n4r\") pod \"monitoring-plugin-76b68cc874-dnrrb\" (UID: \"2739c907-b011-4813-b88a-db69ed5688b1\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:25.729278 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.729197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2739c907-b011-4813-b88a-db69ed5688b1-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-dnrrb\" (UID: \"2739c907-b011-4813-b88a-db69ed5688b1\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:25.731613 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.731584 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2739c907-b011-4813-b88a-db69ed5688b1-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-dnrrb\" (UID: \"2739c907-b011-4813-b88a-db69ed5688b1\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:25.737320 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.737296 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7n4r\" (UniqueName: \"kubernetes.io/projected/2739c907-b011-4813-b88a-db69ed5688b1-kube-api-access-x7n4r\") pod \"monitoring-plugin-76b68cc874-dnrrb\" (UID: \"2739c907-b011-4813-b88a-db69ed5688b1\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:25.844196 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.844158 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:25.968411 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:25.968378 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb"] Jun 08 12:11:25.971425 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:11:25.971392 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2739c907_b011_4813_b88a_db69ed5688b1.slice/crio-d149eb4c4c971867f9cbd0006b88fd453dfb454a50d7a09287a15bfe7e873034 WatchSource:0}: Error finding container d149eb4c4c971867f9cbd0006b88fd453dfb454a50d7a09287a15bfe7e873034: Status 404 returned error can't find the container with id d149eb4c4c971867f9cbd0006b88fd453dfb454a50d7a09287a15bfe7e873034 Jun 08 12:11:26.107237 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.107207 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:26.159691 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.159659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:26.267409 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.267364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:26.329303 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.329265 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:26.664586 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.664551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" event={"ID":"3016476f-e848-49db-89be-0032c86f0c3a","Type":"ContainerStarted","Data":"6fd0c12ddc055f8abdeb68c2762fa74abd0c1a6c03a648ae2f419ec89306a662"} Jun 08 12:11:26.665057 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.664624 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" event={"ID":"3016476f-e848-49db-89be-0032c86f0c3a","Type":"ContainerStarted","Data":"e32c82926cdfdbabfb36298b5d6f071a2cda3e2980481f15ad215ad81a5ebceb"} Jun 08 12:11:26.665057 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.664667 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" event={"ID":"3016476f-e848-49db-89be-0032c86f0c3a","Type":"ContainerStarted","Data":"fb3dac49bc04a5d399098b1c557f030671b68bbe1fc70f1c4dd9a67b3546b894"} Jun 08 12:11:26.665057 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.664715 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:26.665740 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.665710 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" event={"ID":"2739c907-b011-4813-b88a-db69ed5688b1","Type":"ContainerStarted","Data":"d149eb4c4c971867f9cbd0006b88fd453dfb454a50d7a09287a15bfe7e873034"} Jun 08 12:11:26.692085 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.692041 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" podStartSLOduration=1.76948764 podStartE2EDuration="4.692027763s" totalImagesPullingTime="2.922540123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:26.69136149 +0000 UTC m=+249.331202936" watchObservedRunningTime="2026-06-08 12:11:26.692027763 +0000 UTC m=+249.331869197" Jun 08 12:11:26.986458 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.986427 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:11:26.990661 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.990636 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:26.993380 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.991796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:26.993380 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.993075 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 12:11:26.993380 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.993378 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 12:11:26.993584 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.993575 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 12:11:26.993789 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.993770 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 12:11:26.993860 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.993809 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vrgll\"" Jun 08 12:11:26.994357 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.994342 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 12:11:26.995328 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.995223 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-66akh037jjmvs\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.995670 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.996258 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.996710 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.996762 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:26.999969 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.000221 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.000305 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.000352 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.000435 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 12:11:27.000970 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.000606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 12:11:27.007365 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.007342 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:11:27.142294 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142294 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142295 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csfml\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-kube-api-access-csfml\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142497 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142317 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142497 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142497 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142596 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142596 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-config\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142596 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142596 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142580 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142838 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142838 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142838 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142735 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142838 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142778 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.142838 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142822 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.143064 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142840 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.143064 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142865 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.143064 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142964 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.143064 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.142987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.143064 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.143008 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.143064 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.143027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.243900 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.243814 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.243900 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.243852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.243900 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.243894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.243917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csfml\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-kube-api-access-csfml\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.243958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.243981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244013 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244071 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-config\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244637 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244205 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244637 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244637 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244637 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244637 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244637 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.244637 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.244449 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.245171 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.245141 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.245273 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.245200 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.245677 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.245367 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.245773 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.245720 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.249005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.248020 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.249005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.248116 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.249005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.248495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.249005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.248692 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.249512 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.249482 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.250335 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.250309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.250651 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.250626 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.250743 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.250669 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.251113 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.251088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.251904 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.251879 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.252306 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.252266 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.252386 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.252314 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.252963 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.252918 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.253461 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.253441 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.254136 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.254118 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-config\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.254654 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.254628 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csfml\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-kube-api-access-csfml\") pod \"prometheus-k8s-0\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.308604 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.308572 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:27.469293 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:27.469252 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:11:27.728291 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:11:27.728251 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a047233_f12e_4095_9402_02014bd8b915.slice/crio-2adaa73a0f793908f4eeb4b33d194aef93d43c336da740b6ca5999101add5bbe WatchSource:0}: Error finding container 2adaa73a0f793908f4eeb4b33d194aef93d43c336da740b6ca5999101add5bbe: Status 404 returned error can't find the container with id 2adaa73a0f793908f4eeb4b33d194aef93d43c336da740b6ca5999101add5bbe Jun 08 12:11:28.675391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:28.675350 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerStarted","Data":"2adaa73a0f793908f4eeb4b33d194aef93d43c336da740b6ca5999101add5bbe"} Jun 08 12:11:28.676987 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:28.676933 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" event={"ID":"2739c907-b011-4813-b88a-db69ed5688b1","Type":"ContainerStarted","Data":"d2251dd50a966e91dbf24ffccb0f899c86261dfee5a316d66fbf8874ffa9dfb7"} Jun 08 12:11:28.677198 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:28.677177 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:28.682427 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:28.682404 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" Jun 08 12:11:28.695575 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:28.695517 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-dnrrb" podStartSLOduration=1.893571452 podStartE2EDuration="3.695502482s" totalImagesPullingTime="1.80193103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:28.693953572 +0000 UTC m=+251.333795021" watchObservedRunningTime="2026-06-08 12:11:28.695502482 +0000 UTC m=+251.335343916" Jun 08 12:11:29.609483 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.609440 2580 patch_prober.go:28] interesting pod/image-registry-7b68d8d847-bg5jw container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 12:11:29.609839 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.609509 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" podUID="9e12cb94-4c4e-405b-876a-8cab95b9c907" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 12:11:29.665648 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.665610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:11:29.667888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.667861 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd9e9d8-9609-43d5-a45c-e58816b43118-metrics-certs\") pod \"network-metrics-daemon-wr6xl\" (UID: \"bfd9e9d8-9609-43d5-a45c-e58816b43118\") " pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:11:29.681636 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.681600 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a047233-f12e-4095-9402-02014bd8b915" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" exitCode=0 Jun 08 12:11:29.681796 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.681688 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerDied","Data":"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0"} Jun 08 12:11:29.759584 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.759550 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gr2v8\"" Jun 08 12:11:29.766821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.766786 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wr6xl" Jun 08 12:11:29.889238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:29.889041 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wr6xl"] Jun 08 12:11:29.891369 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:11:29.891329 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd9e9d8_9609_43d5_a45c_e58816b43118.slice/crio-6362320b8e42a0849fe5b0a597693b4d1b8e6be9f292aa61810cde27249aeb90 WatchSource:0}: Error finding container 6362320b8e42a0849fe5b0a597693b4d1b8e6be9f292aa61810cde27249aeb90: Status 404 returned error can't find the container with id 6362320b8e42a0849fe5b0a597693b4d1b8e6be9f292aa61810cde27249aeb90 Jun 08 12:11:30.687991 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:30.687901 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wr6xl" event={"ID":"bfd9e9d8-9609-43d5-a45c-e58816b43118","Type":"ContainerStarted","Data":"6362320b8e42a0849fe5b0a597693b4d1b8e6be9f292aa61810cde27249aeb90"} Jun 08 12:11:31.694383 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:31.694347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wr6xl" event={"ID":"bfd9e9d8-9609-43d5-a45c-e58816b43118","Type":"ContainerStarted","Data":"b761376e71941ecb18afea9468e335646f11156cde60eb4e02b5561f4c576620"} Jun 08 12:11:31.694848 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:31.694417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wr6xl" event={"ID":"bfd9e9d8-9609-43d5-a45c-e58816b43118","Type":"ContainerStarted","Data":"4326e7fad3fe3094d7ad5b40460c802da53f3d290b3c4936fc7c7ac113902f7a"} Jun 08 12:11:31.711563 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:31.711508 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wr6xl" podStartSLOduration=253.737384516 podStartE2EDuration="4m14.711491266s" totalImagesPullingTime="974.10675ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:31.710297119 +0000 UTC m=+254.350138577" watchObservedRunningTime="2026-06-08 12:11:31.711491266 +0000 UTC m=+254.351332699" Jun 08 12:11:31.801434 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:31.801391 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:31.863782 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:31.863753 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:32.677526 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:32.677483 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7486f69c5-24lzt" Jun 08 12:11:32.855023 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:32.854992 2580 scope.go:117] "RemoveContainer" containerID="2b52ab0746a01549750fbdc0eb52266791982f781bde1e6bff0a8fdf76b73e98" Jun 08 12:11:33.706007 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.705893 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerStarted","Data":"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a"} Jun 08 12:11:33.706155 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.706005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerStarted","Data":"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e"} Jun 08 12:11:33.706155 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.706052 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerStarted","Data":"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8"} Jun 08 12:11:33.706155 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.706095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerStarted","Data":"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99"} Jun 08 12:11:33.706155 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.706135 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerStarted","Data":"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e"} Jun 08 12:11:33.706364 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.706176 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerStarted","Data":"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b"} Jun 08 12:11:33.707987 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.707967 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:11:33.708106 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.708062 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" event={"ID":"cacc1679-4496-434a-b777-29c9f516ac87","Type":"ContainerStarted","Data":"421647d2967fd3f276c45d748023c2274e2834e080db7cb8062e2d7c12900651"} Jun 08 12:11:33.708343 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.708326 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:11:33.713245 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.713224 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" Jun 08 12:11:33.738530 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.738467 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.251240844 podStartE2EDuration="7.738444162s" totalImagesPullingTime="4.465159296s" totalInitContainerRuntime="22.044022ms" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:33.736071573 +0000 UTC m=+256.375913034" watchObservedRunningTime="2026-06-08 12:11:33.738444162 +0000 UTC m=+256.378285597" Jun 08 12:11:33.754084 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.754039 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-668fdbf85c-kvr8v" podStartSLOduration=52.703453524 podStartE2EDuration="54.754020185s" totalImagesPullingTime="2.050566661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:33.753850614 +0000 UTC m=+256.393692062" watchObservedRunningTime="2026-06-08 12:11:33.754020185 +0000 UTC m=+256.393861619" Jun 08 12:11:33.882526 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.882491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:33.981486 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:33.981457 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:34.041039 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:34.041009 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:34.364235 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:34.364150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:34.426775 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:34.426746 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:34.624070 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:34.623989 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" podUID="9e12cb94-4c4e-405b-876a-8cab95b9c907" containerName="registry" containerID="cri-o://d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b" gracePeriod=30 Jun 08 12:11:34.645999 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:34.645969 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:34.701849 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:34.701822 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:34.766810 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:34.766780 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:34.860135 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:34.860106 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:11:35.018738 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.018709 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-trusted-ca\") pod \"9e12cb94-4c4e-405b-876a-8cab95b9c907\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " Jun 08 12:11:35.019164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.018752 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-certificates\") pod \"9e12cb94-4c4e-405b-876a-8cab95b9c907\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " Jun 08 12:11:35.019164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.018805 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-image-registry-private-configuration\") pod \"9e12cb94-4c4e-405b-876a-8cab95b9c907\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " Jun 08 12:11:35.019164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.018837 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49pb6\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-kube-api-access-49pb6\") pod \"9e12cb94-4c4e-405b-876a-8cab95b9c907\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " Jun 08 12:11:35.019164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.018880 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e12cb94-4c4e-405b-876a-8cab95b9c907-ca-trust-extracted\") pod \"9e12cb94-4c4e-405b-876a-8cab95b9c907\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " Jun 08 12:11:35.019164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.018912 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") pod \"9e12cb94-4c4e-405b-876a-8cab95b9c907\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " Jun 08 12:11:35.019164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.018984 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-installation-pull-secrets\") pod \"9e12cb94-4c4e-405b-876a-8cab95b9c907\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " Jun 08 12:11:35.019164 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.019016 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-bound-sa-token\") pod \"9e12cb94-4c4e-405b-876a-8cab95b9c907\" (UID: \"9e12cb94-4c4e-405b-876a-8cab95b9c907\") " Jun 08 12:11:35.019509 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.019233 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9e12cb94-4c4e-405b-876a-8cab95b9c907" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:11:35.019509 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.019337 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9e12cb94-4c4e-405b-876a-8cab95b9c907" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:11:35.021367 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.021318 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9e12cb94-4c4e-405b-876a-8cab95b9c907" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:11:35.021509 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.021477 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-kube-api-access-49pb6" (OuterVolumeSpecName: "kube-api-access-49pb6") pod "9e12cb94-4c4e-405b-876a-8cab95b9c907" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907"). InnerVolumeSpecName "kube-api-access-49pb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:11:35.021509 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.021478 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9e12cb94-4c4e-405b-876a-8cab95b9c907" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:11:35.021638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.021523 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9e12cb94-4c4e-405b-876a-8cab95b9c907" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:11:35.021687 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.021669 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9e12cb94-4c4e-405b-876a-8cab95b9c907" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:11:35.027358 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.027335 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e12cb94-4c4e-405b-876a-8cab95b9c907-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9e12cb94-4c4e-405b-876a-8cab95b9c907" (UID: "9e12cb94-4c4e-405b-876a-8cab95b9c907"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:11:35.120502 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.120476 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e12cb94-4c4e-405b-876a-8cab95b9c907-ca-trust-extracted\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:11:35.120502 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.120499 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:11:35.120634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.120509 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-installation-pull-secrets\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:11:35.120634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.120518 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-bound-sa-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:11:35.120634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.120526 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-trusted-ca\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:11:35.120634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.120534 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e12cb94-4c4e-405b-876a-8cab95b9c907-registry-certificates\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:11:35.120634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.120543 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e12cb94-4c4e-405b-876a-8cab95b9c907-image-registry-private-configuration\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:11:35.120634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.120552 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49pb6\" (UniqueName: \"kubernetes.io/projected/9e12cb94-4c4e-405b-876a-8cab95b9c907-kube-api-access-49pb6\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:11:35.561133 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.561098 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:35.716901 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.716864 2580 generic.go:356] "Generic (PLEG): container finished" podID="9e12cb94-4c4e-405b-876a-8cab95b9c907" containerID="d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b" exitCode=0 Jun 08 12:11:35.717069 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.716914 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" Jun 08 12:11:35.717069 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.716961 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" event={"ID":"9e12cb94-4c4e-405b-876a-8cab95b9c907","Type":"ContainerDied","Data":"d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b"} Jun 08 12:11:35.717069 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.717021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b68d8d847-bg5jw" event={"ID":"9e12cb94-4c4e-405b-876a-8cab95b9c907","Type":"ContainerDied","Data":"e99fad0485041349432d5ea2a1349ce09fa88386b3e3cfac081b89b10d808ae3"} Jun 08 12:11:35.717069 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.717058 2580 scope.go:117] "RemoveContainer" containerID="d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b" Jun 08 12:11:35.725863 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.725845 2580 scope.go:117] "RemoveContainer" containerID="d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b" Jun 08 12:11:35.726168 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:11:35.726147 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b\": container with ID starting with d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b not found: ID does not exist" containerID="d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b" Jun 08 12:11:35.726227 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.726176 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b"} err="failed to get container status \"d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b\": rpc error: code = NotFound desc = could not find container \"d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b\": container with ID starting with d62f1fa7d9dca63cbc17d6fd123b43a33ca5bdf26777542a29f43f9e22ec112b not found: ID does not exist" Jun 08 12:11:35.741430 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.741404 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b68d8d847-bg5jw"] Jun 08 12:11:35.748919 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.748898 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b68d8d847-bg5jw"] Jun 08 12:11:35.766490 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.766467 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:35.821563 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.821494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:35.859191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.859167 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e12cb94-4c4e-405b-876a-8cab95b9c907" path="/var/lib/kubelet/pods/9e12cb94-4c4e-405b-876a-8cab95b9c907/volumes" Jun 08 12:11:35.973833 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:35.973804 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:37.309071 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:37.309027 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:11:39.846697 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:39.846660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:39.947819 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:39.947790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:44.208866 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:44.208830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:44.287993 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:44.287963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:48.998902 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:48.998870 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:49.051407 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:49.051378 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:52.389596 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:52.389562 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:52.440770 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:52.440740 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:56.467385 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:56.467346 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:56.519699 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:56.519665 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:11:59.368785 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:11:59.368746 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8gqn6" podUID="55247bf6-9e28-41a7-b925-b8e43914ef45" Jun 08 12:11:59.807537 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:11:59.807507 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8gqn6" Jun 08 12:12:01.146124 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:01.146085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:01.246997 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:01.246965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:01.815840 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:01.815757 2580 generic.go:356] "Generic (PLEG): container finished" podID="fdecc9cb-450e-41a7-a3b1-4f656f24fed0" containerID="e3b5dcf2c4f0dcbe9e4ed5ae0519269fbe60f92e24d9bc300a2fcc893ffb3742" exitCode=255 Jun 08 12:12:01.816020 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:01.815832 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" event={"ID":"fdecc9cb-450e-41a7-a3b1-4f656f24fed0","Type":"ContainerDied","Data":"e3b5dcf2c4f0dcbe9e4ed5ae0519269fbe60f92e24d9bc300a2fcc893ffb3742"} Jun 08 12:12:01.816020 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:01.815889 2580 scope.go:117] "RemoveContainer" containerID="434ee0b034dda347a31fb02ac463af3f6756845838220eef49fe181a52e68285" Jun 08 12:12:01.822221 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:01.822203 2580 scope.go:117] "RemoveContainer" containerID="e3b5dcf2c4f0dcbe9e4ed5ae0519269fbe60f92e24d9bc300a2fcc893ffb3742" Jun 08 12:12:01.822373 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:01.822357 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-7f8774f974-76zgv_open-cluster-management-agent-addon(fdecc9cb-450e-41a7-a3b1-4f656f24fed0)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" podUID="fdecc9cb-450e-41a7-a3b1-4f656f24fed0" Jun 08 12:12:02.651743 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:02.651709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:12:02.654051 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:02.654031 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55247bf6-9e28-41a7-b925-b8e43914ef45-metrics-tls\") pod \"dns-default-8gqn6\" (UID: \"55247bf6-9e28-41a7-b925-b8e43914ef45\") " pod="openshift-dns/dns-default-8gqn6" Jun 08 12:12:02.811173 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:02.811142 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6km9q\"" Jun 08 12:12:02.818638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:02.818606 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8gqn6" Jun 08 12:12:02.948042 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:02.947962 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8gqn6"] Jun 08 12:12:02.951849 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:12:02.951823 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55247bf6_9e28_41a7_b925_b8e43914ef45.slice/crio-4f8500c6a05104285fe113fd8692c8dd506a8260b348ec65277f63fe12936f86 WatchSource:0}: Error finding container 4f8500c6a05104285fe113fd8692c8dd506a8260b348ec65277f63fe12936f86: Status 404 returned error can't find the container with id 4f8500c6a05104285fe113fd8692c8dd506a8260b348ec65277f63fe12936f86 Jun 08 12:12:03.828234 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:03.828197 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8gqn6" event={"ID":"55247bf6-9e28-41a7-b925-b8e43914ef45","Type":"ContainerStarted","Data":"4f8500c6a05104285fe113fd8692c8dd506a8260b348ec65277f63fe12936f86"} Jun 08 12:12:05.397264 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.397223 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" podUID="9ae6b08f-8528-497b-afbc-83f62e504b14" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused" Jun 08 12:12:05.838333 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.838299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8gqn6" event={"ID":"55247bf6-9e28-41a7-b925-b8e43914ef45","Type":"ContainerStarted","Data":"9b543a7132d26057e7035823a688bfb3852495bcf86649629fb24918d8ef1ed1"} Jun 08 12:12:05.838521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.838375 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8gqn6" event={"ID":"55247bf6-9e28-41a7-b925-b8e43914ef45","Type":"ContainerStarted","Data":"b309b5d067087adb30574849eed50daf35fde96146c1aa0e460c69872fb80aad"} Jun 08 12:12:05.838521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.838410 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8gqn6" Jun 08 12:12:05.840068 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.840045 2580 generic.go:356] "Generic (PLEG): container finished" podID="9ae6b08f-8528-497b-afbc-83f62e504b14" containerID="d3f3dc796b3fe076efc5551dde13cb053bce254e750c1428d870d03549620a4f" exitCode=1 Jun 08 12:12:05.840167 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.840083 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" event={"ID":"9ae6b08f-8528-497b-afbc-83f62e504b14","Type":"ContainerDied","Data":"d3f3dc796b3fe076efc5551dde13cb053bce254e750c1428d870d03549620a4f"} Jun 08 12:12:05.840167 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.840119 2580 scope.go:117] "RemoveContainer" containerID="07c042bba791c5b67994a23a039f05a10da0d11f574325d7ee054fb1953f8e28" Jun 08 12:12:05.840388 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.840371 2580 scope.go:117] "RemoveContainer" containerID="d3f3dc796b3fe076efc5551dde13cb053bce254e750c1428d870d03549620a4f" Jun 08 12:12:05.840545 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:05.840528 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-fc8bb9cf7-mjrcf_open-cluster-management-agent-addon(9ae6b08f-8528-497b-afbc-83f62e504b14)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" podUID="9ae6b08f-8528-497b-afbc-83f62e504b14" Jun 08 12:12:05.856439 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:05.856386 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8gqn6" podStartSLOduration=251.937030555 podStartE2EDuration="4m13.856368697s" totalImagesPullingTime="1.919338142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:12:05.853857693 +0000 UTC m=+288.493699127" watchObservedRunningTime="2026-06-08 12:12:05.856368697 +0000 UTC m=+288.496210133" Jun 08 12:12:07.622068 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:07.622038 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" Jun 08 12:12:07.622432 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:07.622326 2580 scope.go:117] "RemoveContainer" containerID="e3b5dcf2c4f0dcbe9e4ed5ae0519269fbe60f92e24d9bc300a2fcc893ffb3742" Jun 08 12:12:07.622504 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:07.622481 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-7f8774f974-76zgv_open-cluster-management-agent-addon(fdecc9cb-450e-41a7-a3b1-4f656f24fed0)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" podUID="fdecc9cb-450e-41a7-a3b1-4f656f24fed0" Jun 08 12:12:07.646116 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:07.646084 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:12:07.646406 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:07.646393 2580 scope.go:117] "RemoveContainer" containerID="d3f3dc796b3fe076efc5551dde13cb053bce254e750c1428d870d03549620a4f" Jun 08 12:12:07.646535 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:07.646520 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-fc8bb9cf7-mjrcf_open-cluster-management-agent-addon(9ae6b08f-8528-497b-afbc-83f62e504b14)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" podUID="9ae6b08f-8528-497b-afbc-83f62e504b14" Jun 08 12:12:10.646729 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:10.646697 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:10.711000 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:10.710930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:12.911737 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:12.911701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:13.015424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:13.015389 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:13.940384 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:13.940354 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:15.372991 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:15.372960 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:15.397004 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:15.396979 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:12:15.397337 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:15.397324 2580 scope.go:117] "RemoveContainer" containerID="d3f3dc796b3fe076efc5551dde13cb053bce254e750c1428d870d03549620a4f" Jun 08 12:12:15.847531 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:15.847506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8gqn6" Jun 08 12:12:15.883914 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:15.883870 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" event={"ID":"9ae6b08f-8528-497b-afbc-83f62e504b14","Type":"ContainerStarted","Data":"fe29a5e36ba4f1301d29303721008b21040f089e4d955facfae023d3b7546c20"} Jun 08 12:12:15.884241 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:15.884218 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:12:15.885996 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:15.885917 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fc8bb9cf7-mjrcf" Jun 08 12:12:17.801084 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:17.801051 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:12:17.801519 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:17.801114 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:12:17.805602 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:17.805581 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:12:17.805912 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:17.805896 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:12:17.812458 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:17.812438 2580 kubelet.go:1648] "Image garbage collection succeeded" Jun 08 12:12:18.826821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:18.826790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:18.929510 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:18.929476 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:19.002874 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:19.002844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:19.515548 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:19.515511 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:19.617535 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:19.617503 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:19.862812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:19.862736 2580 scope.go:117] "RemoveContainer" containerID="e3b5dcf2c4f0dcbe9e4ed5ae0519269fbe60f92e24d9bc300a2fcc893ffb3742" Jun 08 12:12:20.249262 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:20.249231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:20.311027 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:20.310990 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:20.469315 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:20.469282 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:20.533410 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:20.533330 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:20.904321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:20.904243 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f8774f974-76zgv" event={"ID":"fdecc9cb-450e-41a7-a3b1-4f656f24fed0","Type":"ContainerStarted","Data":"ec833adc06b99e838159e4c72f64aca89687d17c345301359cd2617a3909fc47"} Jun 08 12:12:24.835864 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:24.835832 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:24.952804 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:24.952773 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:27.309358 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:27.309328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:27.329557 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:27.329535 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:27.943896 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:27.943871 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:29.394110 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:29.394078 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:29.492997 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:29.492962 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:29.595413 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:29.595378 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:36.855409 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:36.855361 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:36.937444 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:36.937413 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:38.083808 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:38.083774 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:38.139076 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:38.139045 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:38.328815 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:38.328785 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:38.429859 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:38.429787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:38.530236 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:38.530207 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:38.597797 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:38.597769 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:40.472399 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:40.472362 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:41.471286 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:41.471251 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:41.535333 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:41.535289 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:43.295419 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:43.295384 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:43.385532 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:43.385496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:45.533399 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.533363 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:12:45.533794 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.533671 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="prometheus" containerID="cri-o://e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" gracePeriod=600 Jun 08 12:12:45.533794 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.533720 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" gracePeriod=600 Jun 08 12:12:45.533794 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.533726 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="thanos-sidecar" containerID="cri-o://12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" gracePeriod=600 Jun 08 12:12:45.533794 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.533727 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="config-reloader" containerID="cri-o://6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" gracePeriod=600 Jun 08 12:12:45.534063 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.533689 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy" containerID="cri-o://cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" gracePeriod=600 Jun 08 12:12:45.534063 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.533732 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy-web" containerID="cri-o://12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" gracePeriod=600 Jun 08 12:12:45.773107 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.773084 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:45.898691 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898619 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-config\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898691 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898658 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-grpc-tls\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898691 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898680 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898703 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-trusted-ca-bundle\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898745 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898772 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-config-out\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898796 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-metrics-client-ca\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898814 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-2\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898829 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-tls-assets\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898847 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csfml\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-kube-api-access-csfml\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898879 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-db\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.898962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898922 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-web-config\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.898975 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-thanos-prometheus-http-client-file\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899004 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-1\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899047 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-tls\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899077 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-kubelet-serving-ca-bundle\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899112 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-0\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899127 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899145 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-serving-certs-ca-bundle\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899172 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-metrics-client-certs\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899205 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-kube-rbac-proxy\") pod \"0a047233-f12e-4095-9402-02014bd8b915\" (UID: \"0a047233-f12e-4095-9402-02014bd8b915\") " Jun 08 12:12:45.899877 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899452 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.900522 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899169 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:12:45.900522 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899405 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:12:45.900522 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.899414 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:12:45.900522 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.900468 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:12:45.900819 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.900752 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:12:45.901192 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.901164 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:12:45.901643 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.901412 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.901643 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.901471 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:12:45.902157 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.902127 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-config-out" (OuterVolumeSpecName: "config-out") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:12:45.902238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.902208 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.902475 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.902448 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.903347 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.903320 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-config" (OuterVolumeSpecName: "config") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.903441 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.903371 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.903441 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.903396 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:12:45.903441 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.903401 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.903628 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.903612 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.903762 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.903747 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-kube-api-access-csfml" (OuterVolumeSpecName: "kube-api-access-csfml") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "kube-api-access-csfml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:12:45.904206 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.904187 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.913673 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.913652 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-web-config" (OuterVolumeSpecName: "web-config") pod "0a047233-f12e-4095-9402-02014bd8b915" (UID: "0a047233-f12e-4095-9402-02014bd8b915"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:12:45.996864 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.996825 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a047233-f12e-4095-9402-02014bd8b915" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" exitCode=0 Jun 08 12:12:45.996864 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.996851 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a047233-f12e-4095-9402-02014bd8b915" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" exitCode=0 Jun 08 12:12:45.996864 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.996862 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a047233-f12e-4095-9402-02014bd8b915" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" exitCode=0 Jun 08 12:12:45.996864 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.996871 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a047233-f12e-4095-9402-02014bd8b915" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" exitCode=0 Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.996880 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a047233-f12e-4095-9402-02014bd8b915" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" exitCode=0 Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.996888 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a047233-f12e-4095-9402-02014bd8b915" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" exitCode=0 Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.996902 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.996907 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerDied","Data":"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a"} Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.997009 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerDied","Data":"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e"} Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.997041 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerDied","Data":"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8"} Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.997075 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerDied","Data":"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99"} Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.997078 2580 scope.go:117] "RemoveContainer" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" Jun 08 12:12:45.997114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.997112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerDied","Data":"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e"} Jun 08 12:12:45.997416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.997152 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerDied","Data":"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b"} Jun 08 12:12:45.997416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.997196 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a047233-f12e-4095-9402-02014bd8b915","Type":"ContainerDied","Data":"2adaa73a0f793908f4eeb4b33d194aef93d43c336da740b6ca5999101add5bbe"} Jun 08 12:12:45.999757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999739 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-db\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999758 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-web-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999769 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999780 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999789 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999797 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999805 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999814 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999823 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-metrics-client-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999832 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-kube-rbac-proxy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999840 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999849 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-grpc-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999858 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999867 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a047233-f12e-4095-9402-02014bd8b915-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999875 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a047233-f12e-4095-9402-02014bd8b915-config-out\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999883 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-configmap-metrics-client-ca\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:45.999888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999890 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a047233-f12e-4095-9402-02014bd8b915-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:46.000454 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999900 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-tls-assets\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:46.000454 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:45.999908 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csfml\" (UniqueName: \"kubernetes.io/projected/0a047233-f12e-4095-9402-02014bd8b915-kube-api-access-csfml\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:12:46.005246 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.005227 2580 scope.go:117] "RemoveContainer" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" Jun 08 12:12:46.011620 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.011604 2580 scope.go:117] "RemoveContainer" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" Jun 08 12:12:46.019537 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.019518 2580 scope.go:117] "RemoveContainer" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" Jun 08 12:12:46.025783 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.025763 2580 scope.go:117] "RemoveContainer" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" Jun 08 12:12:46.032243 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.032225 2580 scope.go:117] "RemoveContainer" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" Jun 08 12:12:46.032523 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.032504 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:12:46.039671 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.039092 2580 scope.go:117] "RemoveContainer" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" Jun 08 12:12:46.040806 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.040789 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:12:46.045621 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.045607 2580 scope.go:117] "RemoveContainer" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" Jun 08 12:12:46.045852 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:46.045835 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": container with ID starting with 5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a not found: ID does not exist" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" Jun 08 12:12:46.045896 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.045859 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a"} err="failed to get container status \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": rpc error: code = NotFound desc = could not find container \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": container with ID starting with 5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a not found: ID does not exist" Jun 08 12:12:46.045896 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.045873 2580 scope.go:117] "RemoveContainer" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" Jun 08 12:12:46.046099 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:46.046084 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": container with ID starting with cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e not found: ID does not exist" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" Jun 08 12:12:46.046141 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046102 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e"} err="failed to get container status \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": rpc error: code = NotFound desc = could not find container \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": container with ID starting with cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e not found: ID does not exist" Jun 08 12:12:46.046141 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046114 2580 scope.go:117] "RemoveContainer" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" Jun 08 12:12:46.046307 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:46.046293 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": container with ID starting with 12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8 not found: ID does not exist" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" Jun 08 12:12:46.046341 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046311 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8"} err="failed to get container status \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": rpc error: code = NotFound desc = could not find container \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": container with ID starting with 12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8 not found: ID does not exist" Jun 08 12:12:46.046341 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046321 2580 scope.go:117] "RemoveContainer" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" Jun 08 12:12:46.046531 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:46.046516 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": container with ID starting with 12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99 not found: ID does not exist" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" Jun 08 12:12:46.046562 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046534 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99"} err="failed to get container status \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": rpc error: code = NotFound desc = could not find container \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": container with ID starting with 12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99 not found: ID does not exist" Jun 08 12:12:46.046562 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046544 2580 scope.go:117] "RemoveContainer" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" Jun 08 12:12:46.046719 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:46.046703 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": container with ID starting with 6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e not found: ID does not exist" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" Jun 08 12:12:46.046752 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046720 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e"} err="failed to get container status \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": rpc error: code = NotFound desc = could not find container \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": container with ID starting with 6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e not found: ID does not exist" Jun 08 12:12:46.046752 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046732 2580 scope.go:117] "RemoveContainer" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" Jun 08 12:12:46.046875 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:46.046861 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": container with ID starting with e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b not found: ID does not exist" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" Jun 08 12:12:46.046913 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046878 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b"} err="failed to get container status \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": rpc error: code = NotFound desc = could not find container \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": container with ID starting with e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b not found: ID does not exist" Jun 08 12:12:46.046913 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.046889 2580 scope.go:117] "RemoveContainer" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" Jun 08 12:12:46.047109 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:12:46.047091 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": container with ID starting with d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0 not found: ID does not exist" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" Jun 08 12:12:46.047174 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047114 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0"} err="failed to get container status \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": rpc error: code = NotFound desc = could not find container \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": container with ID starting with d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0 not found: ID does not exist" Jun 08 12:12:46.047174 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047129 2580 scope.go:117] "RemoveContainer" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" Jun 08 12:12:46.047325 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047309 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a"} err="failed to get container status \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": rpc error: code = NotFound desc = could not find container \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": container with ID starting with 5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a not found: ID does not exist" Jun 08 12:12:46.047363 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047325 2580 scope.go:117] "RemoveContainer" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" Jun 08 12:12:46.047505 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047488 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e"} err="failed to get container status \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": rpc error: code = NotFound desc = could not find container \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": container with ID starting with cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e not found: ID does not exist" Jun 08 12:12:46.047547 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047506 2580 scope.go:117] "RemoveContainer" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" Jun 08 12:12:46.047736 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047721 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8"} err="failed to get container status \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": rpc error: code = NotFound desc = could not find container \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": container with ID starting with 12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8 not found: ID does not exist" Jun 08 12:12:46.047778 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047737 2580 scope.go:117] "RemoveContainer" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" Jun 08 12:12:46.047935 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047910 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99"} err="failed to get container status \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": rpc error: code = NotFound desc = could not find container \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": container with ID starting with 12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99 not found: ID does not exist" Jun 08 12:12:46.047935 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.047933 2580 scope.go:117] "RemoveContainer" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" Jun 08 12:12:46.048189 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048171 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e"} err="failed to get container status \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": rpc error: code = NotFound desc = could not find container \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": container with ID starting with 6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e not found: ID does not exist" Jun 08 12:12:46.048240 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048190 2580 scope.go:117] "RemoveContainer" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" Jun 08 12:12:46.048395 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048381 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b"} err="failed to get container status \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": rpc error: code = NotFound desc = could not find container \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": container with ID starting with e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b not found: ID does not exist" Jun 08 12:12:46.048439 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048397 2580 scope.go:117] "RemoveContainer" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" Jun 08 12:12:46.048591 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048576 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0"} err="failed to get container status \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": rpc error: code = NotFound desc = could not find container \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": container with ID starting with d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0 not found: ID does not exist" Jun 08 12:12:46.048591 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048590 2580 scope.go:117] "RemoveContainer" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" Jun 08 12:12:46.048775 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048761 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a"} err="failed to get container status \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": rpc error: code = NotFound desc = could not find container \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": container with ID starting with 5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a not found: ID does not exist" Jun 08 12:12:46.048813 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048774 2580 scope.go:117] "RemoveContainer" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" Jun 08 12:12:46.049007 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.048984 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e"} err="failed to get container status \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": rpc error: code = NotFound desc = could not find container \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": container with ID starting with cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e not found: ID does not exist" Jun 08 12:12:46.049067 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049009 2580 scope.go:117] "RemoveContainer" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" Jun 08 12:12:46.049212 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049195 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8"} err="failed to get container status \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": rpc error: code = NotFound desc = could not find container \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": container with ID starting with 12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8 not found: ID does not exist" Jun 08 12:12:46.049260 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049212 2580 scope.go:117] "RemoveContainer" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" Jun 08 12:12:46.049416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049402 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99"} err="failed to get container status \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": rpc error: code = NotFound desc = could not find container \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": container with ID starting with 12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99 not found: ID does not exist" Jun 08 12:12:46.049456 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049419 2580 scope.go:117] "RemoveContainer" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" Jun 08 12:12:46.049625 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049602 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e"} err="failed to get container status \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": rpc error: code = NotFound desc = could not find container \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": container with ID starting with 6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e not found: ID does not exist" Jun 08 12:12:46.049625 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049623 2580 scope.go:117] "RemoveContainer" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" Jun 08 12:12:46.049829 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049811 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b"} err="failed to get container status \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": rpc error: code = NotFound desc = could not find container \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": container with ID starting with e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b not found: ID does not exist" Jun 08 12:12:46.049899 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.049830 2580 scope.go:117] "RemoveContainer" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" Jun 08 12:12:46.050087 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050071 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0"} err="failed to get container status \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": rpc error: code = NotFound desc = could not find container \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": container with ID starting with d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0 not found: ID does not exist" Jun 08 12:12:46.050158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050088 2580 scope.go:117] "RemoveContainer" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" Jun 08 12:12:46.050298 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050274 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a"} err="failed to get container status \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": rpc error: code = NotFound desc = could not find container \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": container with ID starting with 5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a not found: ID does not exist" Jun 08 12:12:46.050362 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050299 2580 scope.go:117] "RemoveContainer" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" Jun 08 12:12:46.050525 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050506 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e"} err="failed to get container status \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": rpc error: code = NotFound desc = could not find container \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": container with ID starting with cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e not found: ID does not exist" Jun 08 12:12:46.050575 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050525 2580 scope.go:117] "RemoveContainer" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" Jun 08 12:12:46.050734 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050714 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8"} err="failed to get container status \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": rpc error: code = NotFound desc = could not find container \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": container with ID starting with 12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8 not found: ID does not exist" Jun 08 12:12:46.050771 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050736 2580 scope.go:117] "RemoveContainer" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" Jun 08 12:12:46.050937 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050921 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99"} err="failed to get container status \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": rpc error: code = NotFound desc = could not find container \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": container with ID starting with 12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99 not found: ID does not exist" Jun 08 12:12:46.050995 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.050938 2580 scope.go:117] "RemoveContainer" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" Jun 08 12:12:46.051169 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051151 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e"} err="failed to get container status \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": rpc error: code = NotFound desc = could not find container \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": container with ID starting with 6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e not found: ID does not exist" Jun 08 12:12:46.051216 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051170 2580 scope.go:117] "RemoveContainer" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" Jun 08 12:12:46.051355 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051339 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b"} err="failed to get container status \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": rpc error: code = NotFound desc = could not find container \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": container with ID starting with e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b not found: ID does not exist" Jun 08 12:12:46.051397 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051355 2580 scope.go:117] "RemoveContainer" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" Jun 08 12:12:46.051568 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051550 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0"} err="failed to get container status \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": rpc error: code = NotFound desc = could not find container \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": container with ID starting with d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0 not found: ID does not exist" Jun 08 12:12:46.051611 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051569 2580 scope.go:117] "RemoveContainer" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" Jun 08 12:12:46.051755 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051737 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a"} err="failed to get container status \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": rpc error: code = NotFound desc = could not find container \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": container with ID starting with 5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a not found: ID does not exist" Jun 08 12:12:46.051832 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051756 2580 scope.go:117] "RemoveContainer" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" Jun 08 12:12:46.051975 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051959 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e"} err="failed to get container status \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": rpc error: code = NotFound desc = could not find container \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": container with ID starting with cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e not found: ID does not exist" Jun 08 12:12:46.052014 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.051975 2580 scope.go:117] "RemoveContainer" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" Jun 08 12:12:46.052166 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052153 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8"} err="failed to get container status \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": rpc error: code = NotFound desc = could not find container \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": container with ID starting with 12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8 not found: ID does not exist" Jun 08 12:12:46.052218 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052166 2580 scope.go:117] "RemoveContainer" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" Jun 08 12:12:46.052373 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052355 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99"} err="failed to get container status \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": rpc error: code = NotFound desc = could not find container \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": container with ID starting with 12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99 not found: ID does not exist" Jun 08 12:12:46.052415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052374 2580 scope.go:117] "RemoveContainer" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" Jun 08 12:12:46.052581 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052564 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e"} err="failed to get container status \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": rpc error: code = NotFound desc = could not find container \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": container with ID starting with 6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e not found: ID does not exist" Jun 08 12:12:46.052581 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052580 2580 scope.go:117] "RemoveContainer" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" Jun 08 12:12:46.052788 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052772 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b"} err="failed to get container status \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": rpc error: code = NotFound desc = could not find container \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": container with ID starting with e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b not found: ID does not exist" Jun 08 12:12:46.052832 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052789 2580 scope.go:117] "RemoveContainer" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" Jun 08 12:12:46.052973 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052958 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0"} err="failed to get container status \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": rpc error: code = NotFound desc = could not find container \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": container with ID starting with d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0 not found: ID does not exist" Jun 08 12:12:46.053023 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.052972 2580 scope.go:117] "RemoveContainer" containerID="5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a" Jun 08 12:12:46.053158 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053141 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a"} err="failed to get container status \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": rpc error: code = NotFound desc = could not find container \"5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a\": container with ID starting with 5cf34b0e8faecb035b9130693764cfcd90335e6e011056150631268d6d0af08a not found: ID does not exist" Jun 08 12:12:46.053198 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053159 2580 scope.go:117] "RemoveContainer" containerID="cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e" Jun 08 12:12:46.053343 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053326 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e"} err="failed to get container status \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": rpc error: code = NotFound desc = could not find container \"cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e\": container with ID starting with cbca9b08881516196da7b3971e44f3a10afe24df1d15ae4698a53026c68fde7e not found: ID does not exist" Jun 08 12:12:46.053385 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053345 2580 scope.go:117] "RemoveContainer" containerID="12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8" Jun 08 12:12:46.053537 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053523 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8"} err="failed to get container status \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": rpc error: code = NotFound desc = could not find container \"12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8\": container with ID starting with 12187df0a0aa4248f3afb1c9c3fe2d7958b3fbb15115ae89848a417f931ca2f8 not found: ID does not exist" Jun 08 12:12:46.053582 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053537 2580 scope.go:117] "RemoveContainer" containerID="12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99" Jun 08 12:12:46.053716 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053699 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99"} err="failed to get container status \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": rpc error: code = NotFound desc = could not find container \"12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99\": container with ID starting with 12adbe2b267f9900bf6fdca61c5ac3fe1bbb59f066a9c26f97fab0b6fd7f9b99 not found: ID does not exist" Jun 08 12:12:46.053755 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053717 2580 scope.go:117] "RemoveContainer" containerID="6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e" Jun 08 12:12:46.053900 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053883 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e"} err="failed to get container status \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": rpc error: code = NotFound desc = could not find container \"6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e\": container with ID starting with 6a2ae4264a3cdf1f52babdcab96e148044b38415655fe53037ee2cf2c396a92e not found: ID does not exist" Jun 08 12:12:46.053964 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.053902 2580 scope.go:117] "RemoveContainer" containerID="e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b" Jun 08 12:12:46.054121 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.054105 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b"} err="failed to get container status \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": rpc error: code = NotFound desc = could not find container \"e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b\": container with ID starting with e234a7c0ffc75245c958f48afe54f467506b65957ab701c47ba2e336199b598b not found: ID does not exist" Jun 08 12:12:46.054163 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.054121 2580 scope.go:117] "RemoveContainer" containerID="d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0" Jun 08 12:12:46.054321 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.054303 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0"} err="failed to get container status \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": rpc error: code = NotFound desc = could not find container \"d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0\": container with ID starting with d45c0263aff91bd05524d47715b75bad30df017c278f89df8efcf7061167e8c0 not found: ID does not exist" Jun 08 12:12:46.076398 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076375 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:12:46.076653 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076641 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e12cb94-4c4e-405b-876a-8cab95b9c907" containerName="registry" Jun 08 12:12:46.076695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076655 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e12cb94-4c4e-405b-876a-8cab95b9c907" containerName="registry" Jun 08 12:12:46.076695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076663 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="init-config-reloader" Jun 08 12:12:46.076695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076669 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="init-config-reloader" Jun 08 12:12:46.076695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076682 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="prometheus" Jun 08 12:12:46.076695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076687 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="prometheus" Jun 08 12:12:46.076695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076692 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy-web" Jun 08 12:12:46.076695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076697 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy-web" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076704 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076711 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076720 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="config-reloader" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076725 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="config-reloader" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076731 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="thanos-sidecar" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076736 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="thanos-sidecar" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076743 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy-thanos" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076748 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy-thanos" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076785 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy-web" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076792 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e12cb94-4c4e-405b-876a-8cab95b9c907" containerName="registry" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076797 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076802 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="kube-rbac-proxy-thanos" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076808 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="prometheus" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076814 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="thanos-sidecar" Jun 08 12:12:46.076897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.076820 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a047233-f12e-4095-9402-02014bd8b915" containerName="config-reloader" Jun 08 12:12:46.080644 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.080627 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.085249 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.085232 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 12:12:46.085496 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.085460 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 12:12:46.085496 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.085487 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 12:12:46.085703 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.085517 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 12:12:46.085703 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.085570 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 12:12:46.085703 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.085581 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 12:12:46.086054 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.086035 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 12:12:46.087264 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.087243 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-66akh037jjmvs\"" Jun 08 12:12:46.087344 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.087332 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 12:12:46.087991 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.087973 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 12:12:46.088760 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.088741 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 12:12:46.088760 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.088760 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 12:12:46.089161 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.088775 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 12:12:46.089161 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.088844 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vrgll\"" Jun 08 12:12:46.089161 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.089013 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 12:12:46.089453 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.089435 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 12:12:46.095499 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.092845 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 12:12:46.107412 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.107389 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:12:46.201454 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201454 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201454 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4tkk\" (UniqueName: \"kubernetes.io/projected/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-kube-api-access-x4tkk\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201454 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201536 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201613 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-config\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201684 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.201712 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.202051 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201723 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.202051 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201746 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.202051 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.202051 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201819 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.202051 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201852 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.202051 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.201869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.302913 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.302891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303018 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.302921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303018 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.302938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303018 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.302980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303018 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303188 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303188 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303188 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303188 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303125 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-config\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303188 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303188 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303264 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303357 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4tkk\" (UniqueName: \"kubernetes.io/projected/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-kube-api-access-x4tkk\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303989 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303595 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303989 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303602 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303989 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303778 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.303989 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.303858 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.304214 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.304178 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.304512 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.304486 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.306211 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.306181 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.306311 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.306199 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.306311 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.306244 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.306418 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.306325 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.306695 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.306646 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.306926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.306903 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.306926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.306922 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.307195 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.307177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.308299 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.308279 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-config\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.308613 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.308593 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.308781 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.308762 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.308821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.308767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.309089 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.309071 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.313319 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.313302 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4tkk\" (UniqueName: \"kubernetes.io/projected/7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce-kube-api-access-x4tkk\") pod \"prometheus-k8s-0\" (UID: \"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.390309 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.390290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:46.543907 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:46.543880 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 12:12:46.555436 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:12:46.555410 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6c1cec_022f_4d5d_bbc1_7f1ddab87fce.slice/crio-c5d079f7630916604f15b59b90eca8f8d0a9323fc23dde3975123a2b6834ff39 WatchSource:0}: Error finding container c5d079f7630916604f15b59b90eca8f8d0a9323fc23dde3975123a2b6834ff39: Status 404 returned error can't find the container with id c5d079f7630916604f15b59b90eca8f8d0a9323fc23dde3975123a2b6834ff39 Jun 08 12:12:47.003745 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:47.003712 2580 generic.go:356] "Generic (PLEG): container finished" podID="7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce" containerID="b6cc7d7dcaa92a50c0c83200418f544698568915a0eb8dbb87ed81737aa316b0" exitCode=0 Jun 08 12:12:47.003916 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:47.003782 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce","Type":"ContainerDied","Data":"b6cc7d7dcaa92a50c0c83200418f544698568915a0eb8dbb87ed81737aa316b0"} Jun 08 12:12:47.003916 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:47.003833 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce","Type":"ContainerStarted","Data":"c5d079f7630916604f15b59b90eca8f8d0a9323fc23dde3975123a2b6834ff39"} Jun 08 12:12:47.859910 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:47.859876 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a047233-f12e-4095-9402-02014bd8b915" path="/var/lib/kubelet/pods/0a047233-f12e-4095-9402-02014bd8b915/volumes" Jun 08 12:12:48.011692 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:48.011664 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce","Type":"ContainerStarted","Data":"45fdfcefb90add4d465f7dcc3a76bc4dcdc1fcb30623b1e7b3887829611bdade"} Jun 08 12:12:48.011845 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:48.011726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce","Type":"ContainerStarted","Data":"e0bf98812e46bf71f23aca60cbaaf193e32df4dc6b0b815855be033e45176647"} Jun 08 12:12:48.011845 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:48.011752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce","Type":"ContainerStarted","Data":"7b052f25fbe54826a51cbb26861df896024c003c975fd3d8297f8b00410f9e12"} Jun 08 12:12:48.011845 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:48.011775 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce","Type":"ContainerStarted","Data":"db5a840834fbd7eebcc214a8fbbea28c03996280dc6588b0956b12062e99df5a"} Jun 08 12:12:48.011845 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:48.011802 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce","Type":"ContainerStarted","Data":"8c62dc3bfc765476871c382e7701d5d74016cbff4a30e4f27de7699dbb57f899"} Jun 08 12:12:48.011845 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:48.011825 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce","Type":"ContainerStarted","Data":"28ef9b8800407b6e485f7aa199547dae39fd4a6150823dc08a13515361415964"} Jun 08 12:12:48.088553 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:48.088496 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.067782332 podStartE2EDuration="2.088480625s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.698293ms" isStatefulPod=false podCreationTimestamp="2026-06-08 12:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:12:48.080688167 +0000 UTC m=+330.720529613" watchObservedRunningTime="2026-06-08 12:12:48.088480625 +0000 UTC m=+330.728322059" Jun 08 12:12:51.390424 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:51.390390 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:12:56.491168 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:56.491132 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:56.592297 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:56.592269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:12:57.182465 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:12:57.182433 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:04.292199 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:04.292165 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:05.460376 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:05.460344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:05.559732 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:05.559701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:06.079449 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:06.079374 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:06.164645 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:06.164607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:16.766874 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:16.766838 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:16.864210 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:16.864180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:16.920897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:16.920861 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:46.390578 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:46.390543 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:13:46.406202 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:46.406177 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:13:47.249304 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:47.249278 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 12:13:47.945285 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:47.945251 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:13:48.045459 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:13:48.045426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:15:41.045130 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:15:41.045087 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:15:41.120456 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:15:41.120418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:15:41.690308 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:15:41.690272 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:16:24.763535 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:16:24.763495 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:16:24.828992 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:16:24.828959 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:16:25.079130 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:16:25.079033 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:16:25.178963 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:16:25.178910 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:16:25.363401 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:16:25.363316 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:16:25.464307 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:16:25.464275 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:16:58.339663 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:16:58.339624 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:16:58.440969 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:16:58.440924 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:17:09.262069 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:09.262031 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:17:09.363630 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:09.363588 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:17:09.464310 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:09.464274 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:17:17.796959 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:17.796909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:17:17.825000 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:17.824969 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:17:17.825855 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:17.825832 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:17:17.829238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:17.829218 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:17:17.830093 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:17.830079 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:17:25.700413 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:25.700371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:17:25.800456 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:25.800415 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:17:44.469095 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:17:44.469003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:18:20.635851 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:18:20.635813 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:18:20.769086 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:18:20.769050 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:18:20.917324 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:18:20.917240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:18:21.054592 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:18:21.054554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:18:21.287661 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:18:21.287632 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:18:21.610907 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:18:21.610811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:18:40.171141 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:18:40.171084 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:18:42.173618 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:18:42.173581 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:19:42.390222 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:19:42.390180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:19:42.549402 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:19:42.549372 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:19:42.815884 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:19:42.815841 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:19:43.061857 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:19:43.061817 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:19:43.213631 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:19:43.213539 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:20:02.474252 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:20:02.474210 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:03.885602 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:03.885516 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:03.996467 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:03.996431 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:04.096300 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:04.096271 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:04.212780 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:04.212685 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:09.171835 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:09.171790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:25.648660 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:25.648620 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:25.728850 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:25.728812 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:34.388743 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:34.388689 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:46.843906 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:46.843871 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:47.145109 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:47.144994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:47.239002 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:47.238933 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:57.637095 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:57.637054 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:57.738157 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:57.738107 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:57.851396 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:57.851352 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:21:57.961751 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:21:57.961651 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:22:17.853028 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:17.853002 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:22:17.854965 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:17.854924 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:22:17.857436 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:17.857415 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:22:17.859653 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:17.859633 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:22:19.541184 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:19.541145 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:22:20.890072 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:20.890036 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-2zhgx"] Jun 08 12:22:20.893505 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:20.893481 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2zhgx" Jun 08 12:22:20.897615 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:20.897593 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-vtfsk\"" Jun 08 12:22:20.897733 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:20.897594 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jun 08 12:22:20.897733 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:20.897626 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 08 12:22:20.898595 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:20.898575 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 08 12:22:20.906553 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:20.906534 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-2zhgx"] Jun 08 12:22:21.066193 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:21.066159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b594k\" (UniqueName: \"kubernetes.io/projected/a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7-kube-api-access-b594k\") pod \"feast-test-apply-2zhgx\" (UID: \"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7\") " pod="test-ns-feast/feast-test-apply-2zhgx" Jun 08 12:22:21.167089 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:21.167002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b594k\" (UniqueName: \"kubernetes.io/projected/a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7-kube-api-access-b594k\") pod \"feast-test-apply-2zhgx\" (UID: \"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7\") " pod="test-ns-feast/feast-test-apply-2zhgx" Jun 08 12:22:21.177717 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:21.177693 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b594k\" (UniqueName: \"kubernetes.io/projected/a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7-kube-api-access-b594k\") pod \"feast-test-apply-2zhgx\" (UID: \"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7\") " pod="test-ns-feast/feast-test-apply-2zhgx" Jun 08 12:22:21.202621 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:21.202595 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2zhgx" Jun 08 12:22:21.329992 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:21.329969 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-2zhgx"] Jun 08 12:22:21.332140 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:22:21.332114 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9b7dda9_87a8_43fe_af2b_3cbaaa5ac3d7.slice/crio-e7059002e386b9987e8df131d560bbc5efed7b677413c76546dfcbd7b5734e52 WatchSource:0}: Error finding container e7059002e386b9987e8df131d560bbc5efed7b677413c76546dfcbd7b5734e52: Status 404 returned error can't find the container with id e7059002e386b9987e8df131d560bbc5efed7b677413c76546dfcbd7b5734e52 Jun 08 12:22:21.333925 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:21.333908 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 12:22:22.132604 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:22.132564 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2zhgx" event={"ID":"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7","Type":"ContainerStarted","Data":"e7059002e386b9987e8df131d560bbc5efed7b677413c76546dfcbd7b5734e52"} Jun 08 12:22:27.157074 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:27.157033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2zhgx" event={"ID":"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7","Type":"ContainerStarted","Data":"39c544708f6413bac64df2d07fd111742fd164ab6696b29e69646e01b8cf38ef"} Jun 08 12:22:33.180675 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:33.180587 2580 generic.go:356] "Generic (PLEG): container finished" podID="a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" containerID="39c544708f6413bac64df2d07fd111742fd164ab6696b29e69646e01b8cf38ef" exitCode=0 Jun 08 12:22:33.181142 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:33.180664 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2zhgx" event={"ID":"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7","Type":"ContainerDied","Data":"39c544708f6413bac64df2d07fd111742fd164ab6696b29e69646e01b8cf38ef"} Jun 08 12:22:34.186032 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:34.185998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2zhgx" event={"ID":"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7","Type":"ContainerStarted","Data":"b6c3921e13ae0fa02ac5736914c9ad719003a3fd3d8bb474a82bbc54052e576c"} Jun 08 12:22:34.213988 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:34.213919 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-2zhgx" podStartSLOduration=2.7396288269999998 podStartE2EDuration="14.213905299s" totalImagesPullingTime="5.057346806s" totalInitContainerRuntime="6.416929666s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:22:34.213500564 +0000 UTC m=+916.853342001" watchObservedRunningTime="2026-06-08 12:22:34.213905299 +0000 UTC m=+916.853746733" Jun 08 12:22:43.221167 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:43.221128 2580 generic.go:356] "Generic (PLEG): container finished" podID="a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" containerID="b6c3921e13ae0fa02ac5736914c9ad719003a3fd3d8bb474a82bbc54052e576c" exitCode=0 Jun 08 12:22:43.221542 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:43.221160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2zhgx" event={"ID":"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7","Type":"ContainerDied","Data":"b6c3921e13ae0fa02ac5736914c9ad719003a3fd3d8bb474a82bbc54052e576c"} Jun 08 12:22:44.351367 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:44.351347 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2zhgx" Jun 08 12:22:44.456357 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:44.456313 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b594k\" (UniqueName: \"kubernetes.io/projected/a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7-kube-api-access-b594k\") pod \"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7\" (UID: \"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7\") " Jun 08 12:22:44.458443 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:44.458420 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7-kube-api-access-b594k" (OuterVolumeSpecName: "kube-api-access-b594k") pod "a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" (UID: "a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7"). InnerVolumeSpecName "kube-api-access-b594k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:22:44.557456 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:44.557386 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b594k\" (UniqueName: \"kubernetes.io/projected/a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7-kube-api-access-b594k\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:22:45.230339 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:45.230308 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2zhgx" Jun 08 12:22:45.230339 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:45.230317 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2zhgx" event={"ID":"a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7","Type":"ContainerDied","Data":"e7059002e386b9987e8df131d560bbc5efed7b677413c76546dfcbd7b5734e52"} Jun 08 12:22:45.230339 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:45.230348 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7059002e386b9987e8df131d560bbc5efed7b677413c76546dfcbd7b5734e52" Jun 08 12:22:45.412540 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:45.412516 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-2zhgx_a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7/feast-0/0.log" Jun 08 12:22:45.419544 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:22:45.419524 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-2zhgx_a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7/feast-1/0.log" Jun 08 12:23:17.000701 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:23:17.000668 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:23:21.936881 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:23:21.936847 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-2zhgx"] Jun 08 12:23:21.945082 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:23:21.945060 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-2zhgx"] Jun 08 12:23:22.059821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:23:22.059787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:23:23.859290 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:23:23.859257 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" path="/var/lib/kubelet/pods/a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7/volumes" Jun 08 12:23:59.195884 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:23:59.195787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:23:59.293627 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:23:59.293595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:24:00.242685 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:24:00.242638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:25:34.992606 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:25:34.992569 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:25:35.076211 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:25:35.076176 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:25:35.235530 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:25:35.235497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:25:40.742820 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:25:40.742790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:25:46.189715 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:25:46.189685 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:25:46.288817 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:25:46.288783 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:25:46.415907 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:25:46.415875 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:25:46.505133 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:25:46.505103 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:26:08.010799 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:26:08.010767 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:26:12.016627 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:26:12.016593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:26:12.108439 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:26:12.108409 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:26:12.322638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:26:12.322553 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:26:12.423244 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:26:12.423210 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:27:00.395664 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:00.395581 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:27:17.879744 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:17.879714 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:27:17.882195 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:17.882176 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:27:17.883808 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:17.883790 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:27:17.886230 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:17.886211 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:27:53.753999 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:53.753957 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:27:56.010163 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:56.010132 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:27:56.463830 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:56.463748 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:27:56.820526 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:56.820498 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:27:57.010162 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:57.010125 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:27:58.815818 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:27:58.815782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:02.005213 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:02.005180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:02.104524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:02.104490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:02.722717 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:02.722687 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:07.667067 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:07.666987 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:07.760534 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:07.760499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:08.173005 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:08.172977 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:24.553914 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:24.553877 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:25.284780 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.284743 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz"] Jun 08 12:28:25.285091 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.285077 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" containerName="feast-0" Jun 08 12:28:25.285145 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.285092 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" containerName="feast-0" Jun 08 12:28:25.285145 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.285102 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" containerName="feast-1" Jun 08 12:28:25.285145 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.285108 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" containerName="feast-1" Jun 08 12:28:25.285259 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.285157 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9b7dda9-87a8-43fe-af2b-3cbaaa5ac3d7" containerName="feast-1" Jun 08 12:28:25.288543 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.288524 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.297791 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.297774 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 08 12:28:25.298599 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.298580 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jun 08 12:28:25.298685 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.298580 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 08 12:28:25.298685 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.298582 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-zm8sx\"" Jun 08 12:28:25.298685 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.298627 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jun 08 12:28:25.298685 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.298627 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jun 08 12:28:25.298823 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.298614 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jun 08 12:28:25.325649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.325622 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz"] Jun 08 12:28:25.439812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.439780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.439812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.439816 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsvrl\" (UniqueName: \"kubernetes.io/projected/df0c8d86-02d4-420e-a403-a7e59860ed5a-kube-api-access-dsvrl\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.440054 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.439848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.440054 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.439864 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.440054 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.439932 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.440054 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.440001 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c8d86-02d4-420e-a403-a7e59860ed5a-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.541130 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.541049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.541130 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.541087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsvrl\" (UniqueName: \"kubernetes.io/projected/df0c8d86-02d4-420e-a403-a7e59860ed5a-kube-api-access-dsvrl\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.541338 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.541135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.541338 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.541157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.541338 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.541192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.541338 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.541231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c8d86-02d4-420e-a403-a7e59860ed5a-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.541625 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.541593 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c8d86-02d4-420e-a403-a7e59860ed5a-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.543671 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.543645 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.543797 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.543700 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.543797 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.543734 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.543797 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.543778 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.554042 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.554020 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsvrl\" (UniqueName: \"kubernetes.io/projected/df0c8d86-02d4-420e-a403-a7e59860ed5a-kube-api-access-dsvrl\") pod \"feast-simple-feast-setup-75c64c888b-4scpz\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.598291 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.598271 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:25.739655 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.739626 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz"] Jun 08 12:28:25.743006 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:28:25.742979 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf0c8d86_02d4_420e_a403_a7e59860ed5a.slice/crio-09f36122d1885748541a39a09006d2d5dc9b2f6529b14dbe22de839c31402d3d WatchSource:0}: Error finding container 09f36122d1885748541a39a09006d2d5dc9b2f6529b14dbe22de839c31402d3d: Status 404 returned error can't find the container with id 09f36122d1885748541a39a09006d2d5dc9b2f6529b14dbe22de839c31402d3d Jun 08 12:28:25.744697 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.744680 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 12:28:25.859985 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:25.859898 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:26.520268 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:26.520229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerStarted","Data":"09f36122d1885748541a39a09006d2d5dc9b2f6529b14dbe22de839c31402d3d"} Jun 08 12:28:36.840815 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:36.840778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:36.939779 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:36.939747 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:36.999211 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:36.999172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:41.215771 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:41.215735 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:28:41.591603 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:41.591563 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerStarted","Data":"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4"} Jun 08 12:28:45.609221 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:45.609186 2580 generic.go:356] "Generic (PLEG): container finished" podID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerID="7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4" exitCode=0 Jun 08 12:28:45.609558 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:45.609261 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerDied","Data":"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4"} Jun 08 12:28:46.615304 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:46.615268 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerStarted","Data":"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57"} Jun 08 12:28:51.635739 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:51.635699 2580 generic.go:356] "Generic (PLEG): container finished" podID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerID="f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57" exitCode=0 Jun 08 12:28:51.635739 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:51.635740 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerDied","Data":"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57"} Jun 08 12:28:52.646873 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:52.646830 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerStarted","Data":"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b"} Jun 08 12:28:52.647380 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:52.646896 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerStarted","Data":"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58"} Jun 08 12:28:52.647380 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:52.646920 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerStarted","Data":"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809"} Jun 08 12:28:52.647380 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:52.646964 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerStarted","Data":"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4"} Jun 08 12:28:52.674687 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:52.674625 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podStartSLOduration=12.74242685 podStartE2EDuration="27.674602786s" totalImagesPullingTime="14.932175936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:28:52.672259721 +0000 UTC m=+1295.312101452" watchObservedRunningTime="2026-06-08 12:28:52.674602786 +0000 UTC m=+1295.314444225" Jun 08 12:28:55.599551 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.599503 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:55.600065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.599579 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:55.600065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.599601 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:55.600065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.599620 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:55.600065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.599637 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:55.600065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.599654 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:55.600065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.599671 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:55.600065 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.599688 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:55.601552 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.601511 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jun 08 12:28:55.601679 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.601542 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.20:6571: connect: connection refused" Jun 08 12:28:55.601679 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.601554 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jun 08 12:28:55.601679 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:55.601618 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused" Jun 08 12:28:58.599600 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:58.599557 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:58.600138 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:58.599758 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:58.600138 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:58.599967 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:58.600476 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:58.600459 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:58.600579 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:58.600525 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:58.600579 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:58.600578 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:58.604145 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:58.604126 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:28:58.684173 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:28:58.684139 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:29:17.910722 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:29:17.910689 2580 scope.go:117] "RemoveContainer" containerID="39c544708f6413bac64df2d07fd111742fd164ab6696b29e69646e01b8cf38ef" Jun 08 12:29:17.918111 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:29:17.918091 2580 scope.go:117] "RemoveContainer" containerID="b6c3921e13ae0fa02ac5736914c9ad719003a3fd3d8bb474a82bbc54052e576c" Jun 08 12:30:00.270801 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.270697 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:30:00.279716 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.279686 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz"] Jun 08 12:30:00.279935 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.279904 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="registry" containerID="cri-o://7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4" gracePeriod=30 Jun 08 12:30:00.280103 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.279971 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="ui" containerID="cri-o://c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b" gracePeriod=30 Jun 08 12:30:00.280103 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.279994 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="offline" containerID="cri-o://c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58" gracePeriod=30 Jun 08 12:30:00.280235 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.280002 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="online" containerID="cri-o://ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809" gracePeriod=30 Jun 08 12:30:00.371214 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.371180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:30:00.475820 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.475790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:30:00.562136 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.562049 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz"] Jun 08 12:30:00.565617 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.565596 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.567123 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.567100 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:30:00.568380 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.568361 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-dfsqs\"" Jun 08 12:30:00.587331 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.587296 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz"] Jun 08 12:30:00.648998 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.648965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.649183 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.649029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.649183 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.649054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.649183 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.649070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwm6r\" (UniqueName: \"kubernetes.io/projected/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-kube-api-access-bwm6r\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.649183 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.649110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.649183 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.649126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.696099 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.696064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:30:00.750115 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.750075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.750316 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.750164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.750316 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.750195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.750316 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.750218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwm6r\" (UniqueName: \"kubernetes.io/projected/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-kube-api-access-bwm6r\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.750316 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:00.750231 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jun 08 12:30:00.750316 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:00.750307 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls podName:7bf36b67-bcac-486d-a8ae-270ea1dacc6d nodeName:}" failed. No retries permitted until 2026-06-08 12:30:01.250286829 +0000 UTC m=+1363.890128245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls") pod "feast-simple-feast-setup-75c64c888b-g8qzz" (UID: "7bf36b67-bcac-486d-a8ae-270ea1dacc6d") : secret "feast-simple-feast-setup-online-tls" not found Jun 08 12:30:00.750580 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.750441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.750580 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.750485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.750580 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:00.750565 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jun 08 12:30:00.750728 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:00.750625 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls podName:7bf36b67-bcac-486d-a8ae-270ea1dacc6d nodeName:}" failed. No retries permitted until 2026-06-08 12:30:01.250606045 +0000 UTC m=+1363.890447459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls") pod "feast-simple-feast-setup-75c64c888b-g8qzz" (UID: "7bf36b67-bcac-486d-a8ae-270ea1dacc6d") : secret "feast-simple-feast-setup-offline-tls" not found Jun 08 12:30:00.750920 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.750886 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.752807 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.752776 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.752807 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.752801 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.762407 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.762386 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwm6r\" (UniqueName: \"kubernetes.io/projected/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-kube-api-access-bwm6r\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:00.799032 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.798998 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:30:00.911643 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.911552 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:30:00.919201 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.919166 2580 generic.go:356] "Generic (PLEG): container finished" podID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerID="c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b" exitCode=0 Jun 08 12:30:00.919305 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:00.919233 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerDied","Data":"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b"} Jun 08 12:30:01.030245 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.030218 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:30:01.254893 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.254851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:01.255084 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.254998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:01.257216 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.257187 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:01.257331 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.257197 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-g8qzz\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:01.475285 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.475254 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:01.608682 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.608653 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz"] Jun 08 12:30:01.609922 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:30:01.609898 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bf36b67_bcac_486d_a8ae_270ea1dacc6d.slice/crio-896bcb83fab5675b2cdf11839c3e6579f31b8f990d376f6550c6678905fb5328 WatchSource:0}: Error finding container 896bcb83fab5675b2cdf11839c3e6579f31b8f990d376f6550c6678905fb5328: Status 404 returned error can't find the container with id 896bcb83fab5675b2cdf11839c3e6579f31b8f990d376f6550c6678905fb5328 Jun 08 12:30:01.926780 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.926682 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerStarted","Data":"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524"} Jun 08 12:30:01.926780 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.926747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerStarted","Data":"896bcb83fab5675b2cdf11839c3e6579f31b8f990d376f6550c6678905fb5328"} Jun 08 12:30:01.930026 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.929899 2580 generic.go:356] "Generic (PLEG): container finished" podID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerID="ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809" exitCode=0 Jun 08 12:30:01.930026 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:01.929974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerDied","Data":"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809"} Jun 08 12:30:06.953608 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:06.953574 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerID="3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524" exitCode=0 Jun 08 12:30:06.954055 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:06.953653 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerDied","Data":"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524"} Jun 08 12:30:07.961152 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:07.961114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerStarted","Data":"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202"} Jun 08 12:30:08.600511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:08.600467 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jun 08 12:30:08.681707 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:08.681668 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jun 08 12:30:12.992137 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:12.992106 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerID="a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202" exitCode=0 Jun 08 12:30:12.992524 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:12.992176 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerDied","Data":"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202"} Jun 08 12:30:14.002647 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:14.002611 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerStarted","Data":"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113"} Jun 08 12:30:14.003151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:14.002670 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerStarted","Data":"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9"} Jun 08 12:30:14.003151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:14.002691 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerStarted","Data":"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6"} Jun 08 12:30:14.003151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:14.002709 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerStarted","Data":"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03"} Jun 08 12:30:14.054045 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:14.053976 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podStartSLOduration=14.053920748 podStartE2EDuration="14.053920748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:30:14.052063668 +0000 UTC m=+1376.691905129" watchObservedRunningTime="2026-06-08 12:30:14.053920748 +0000 UTC m=+1376.693762182" Jun 08 12:30:16.476186 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:16.476045 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:16.476812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:16.476727 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:16.476812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:16.476770 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:16.476812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:16.476790 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:16.478098 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:16.478064 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Jun 08 12:30:16.478232 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:16.478143 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.21:8816: connect: connection refused" Jun 08 12:30:16.478232 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:16.478154 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.21:6571: connect: connection refused" Jun 08 12:30:16.478232 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:16.478062 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Jun 08 12:30:18.601195 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:18.601103 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jun 08 12:30:18.681919 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:18.681870 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jun 08 12:30:19.477101 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.477067 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.477370 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.477351 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.477485 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.477473 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.477842 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.477815 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.477994 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.477860 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.477994 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.477880 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.478125 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.478033 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.478170 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.478161 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.478333 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.478316 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:19.481718 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:19.481701 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:20.034741 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:20.034707 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:20.037848 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:20.037824 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:30:28.600198 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:28.600151 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jun 08 12:30:28.600627 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:28.600293 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:30:28.680957 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:28.680904 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jun 08 12:30:28.681132 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:28.681012 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:30:30.929046 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:30.929019 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:30:31.012112 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.012079 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-registry-tls\") pod \"df0c8d86-02d4-420e-a403-a7e59860ed5a\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " Jun 08 12:30:31.012268 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.012136 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsvrl\" (UniqueName: \"kubernetes.io/projected/df0c8d86-02d4-420e-a403-a7e59860ed5a-kube-api-access-dsvrl\") pod \"df0c8d86-02d4-420e-a403-a7e59860ed5a\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " Jun 08 12:30:31.012268 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.012159 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c8d86-02d4-420e-a403-a7e59860ed5a-feast-data\") pod \"df0c8d86-02d4-420e-a403-a7e59860ed5a\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " Jun 08 12:30:31.012268 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.012187 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-offline-tls\") pod \"df0c8d86-02d4-420e-a403-a7e59860ed5a\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " Jun 08 12:30:31.012268 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.012204 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-online-tls\") pod \"df0c8d86-02d4-420e-a403-a7e59860ed5a\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " Jun 08 12:30:31.012268 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.012234 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-ui-tls\") pod \"df0c8d86-02d4-420e-a403-a7e59860ed5a\" (UID: \"df0c8d86-02d4-420e-a403-a7e59860ed5a\") " Jun 08 12:30:31.012733 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.012692 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0c8d86-02d4-420e-a403-a7e59860ed5a-feast-data" (OuterVolumeSpecName: "feast-data") pod "df0c8d86-02d4-420e-a403-a7e59860ed5a" (UID: "df0c8d86-02d4-420e-a403-a7e59860ed5a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:30:31.014511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.014477 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "df0c8d86-02d4-420e-a403-a7e59860ed5a" (UID: "df0c8d86-02d4-420e-a403-a7e59860ed5a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:30:31.014511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.014502 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-online-tls" (OuterVolumeSpecName: "online-tls") pod "df0c8d86-02d4-420e-a403-a7e59860ed5a" (UID: "df0c8d86-02d4-420e-a403-a7e59860ed5a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:30:31.014675 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.014514 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0c8d86-02d4-420e-a403-a7e59860ed5a-kube-api-access-dsvrl" (OuterVolumeSpecName: "kube-api-access-dsvrl") pod "df0c8d86-02d4-420e-a403-a7e59860ed5a" (UID: "df0c8d86-02d4-420e-a403-a7e59860ed5a"). InnerVolumeSpecName "kube-api-access-dsvrl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:30:31.014743 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.014722 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "df0c8d86-02d4-420e-a403-a7e59860ed5a" (UID: "df0c8d86-02d4-420e-a403-a7e59860ed5a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:30:31.014803 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.014746 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "df0c8d86-02d4-420e-a403-a7e59860ed5a" (UID: "df0c8d86-02d4-420e-a403-a7e59860ed5a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:30:31.079127 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.079035 2580 generic.go:356] "Generic (PLEG): container finished" podID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerID="c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58" exitCode=137 Jun 08 12:30:31.079127 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.079071 2580 generic.go:356] "Generic (PLEG): container finished" podID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerID="7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4" exitCode=137 Jun 08 12:30:31.079127 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.079097 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" Jun 08 12:30:31.079127 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.079110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerDied","Data":"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58"} Jun 08 12:30:31.079463 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.079157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerDied","Data":"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4"} Jun 08 12:30:31.079463 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.079172 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz" event={"ID":"df0c8d86-02d4-420e-a403-a7e59860ed5a","Type":"ContainerDied","Data":"09f36122d1885748541a39a09006d2d5dc9b2f6529b14dbe22de839c31402d3d"} Jun 08 12:30:31.079463 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.079179 2580 scope.go:117] "RemoveContainer" containerID="c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b" Jun 08 12:30:31.088208 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.088189 2580 scope.go:117] "RemoveContainer" containerID="c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58" Jun 08 12:30:31.095699 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.095681 2580 scope.go:117] "RemoveContainer" containerID="ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809" Jun 08 12:30:31.103244 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.103228 2580 scope.go:117] "RemoveContainer" containerID="7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4" Jun 08 12:30:31.103308 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.103255 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz"] Jun 08 12:30:31.107590 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.107573 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4scpz"] Jun 08 12:30:31.110835 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.110819 2580 scope.go:117] "RemoveContainer" containerID="f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57" Jun 08 12:30:31.113074 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.113055 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c8d86-02d4-420e-a403-a7e59860ed5a-feast-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:30:31.113074 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.113074 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-offline-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:30:31.113213 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.113083 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-online-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:30:31.113213 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.113091 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-ui-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:30:31.113213 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.113098 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df0c8d86-02d4-420e-a403-a7e59860ed5a-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:30:31.113213 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.113106 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsvrl\" (UniqueName: \"kubernetes.io/projected/df0c8d86-02d4-420e-a403-a7e59860ed5a-kube-api-access-dsvrl\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:30:31.117793 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.117784 2580 scope.go:117] "RemoveContainer" containerID="7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4" Jun 08 12:30:31.124638 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.124600 2580 scope.go:117] "RemoveContainer" containerID="c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b" Jun 08 12:30:31.124888 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:31.124867 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b\": container with ID starting with c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b not found: ID does not exist" containerID="c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b" Jun 08 12:30:31.124963 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.124892 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b"} err="failed to get container status \"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b\": rpc error: code = NotFound desc = could not find container \"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b\": container with ID starting with c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b not found: ID does not exist" Jun 08 12:30:31.124963 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.124908 2580 scope.go:117] "RemoveContainer" containerID="c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58" Jun 08 12:30:31.125175 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:31.125157 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58\": container with ID starting with c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58 not found: ID does not exist" containerID="c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58" Jun 08 12:30:31.125249 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.125183 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58"} err="failed to get container status \"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58\": rpc error: code = NotFound desc = could not find container \"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58\": container with ID starting with c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58 not found: ID does not exist" Jun 08 12:30:31.125249 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.125205 2580 scope.go:117] "RemoveContainer" containerID="ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809" Jun 08 12:30:31.125448 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:31.125430 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809\": container with ID starting with ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809 not found: ID does not exist" containerID="ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809" Jun 08 12:30:31.125501 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.125452 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809"} err="failed to get container status \"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809\": rpc error: code = NotFound desc = could not find container \"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809\": container with ID starting with ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809 not found: ID does not exist" Jun 08 12:30:31.125501 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.125468 2580 scope.go:117] "RemoveContainer" containerID="7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4" Jun 08 12:30:31.125684 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:31.125667 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4\": container with ID starting with 7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4 not found: ID does not exist" containerID="7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4" Jun 08 12:30:31.125733 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.125688 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4"} err="failed to get container status \"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4\": rpc error: code = NotFound desc = could not find container \"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4\": container with ID starting with 7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4 not found: ID does not exist" Jun 08 12:30:31.125733 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.125701 2580 scope.go:117] "RemoveContainer" containerID="f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57" Jun 08 12:30:31.125925 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:31.125906 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57\": container with ID starting with f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57 not found: ID does not exist" containerID="f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57" Jun 08 12:30:31.126043 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.125931 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57"} err="failed to get container status \"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57\": rpc error: code = NotFound desc = could not find container \"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57\": container with ID starting with f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57 not found: ID does not exist" Jun 08 12:30:31.126043 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.125974 2580 scope.go:117] "RemoveContainer" containerID="7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4" Jun 08 12:30:31.126226 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:30:31.126209 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4\": container with ID starting with 7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4 not found: ID does not exist" containerID="7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4" Jun 08 12:30:31.126267 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.126233 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4"} err="failed to get container status \"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4\": rpc error: code = NotFound desc = could not find container \"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4\": container with ID starting with 7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4 not found: ID does not exist" Jun 08 12:30:31.126267 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.126245 2580 scope.go:117] "RemoveContainer" containerID="c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b" Jun 08 12:30:31.126455 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.126431 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b"} err="failed to get container status \"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b\": rpc error: code = NotFound desc = could not find container \"c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b\": container with ID starting with c18222af4c73632c2f08147bdc6ee2b3520534aaa8ab90f9f6350e4f70d4044b not found: ID does not exist" Jun 08 12:30:31.126497 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.126457 2580 scope.go:117] "RemoveContainer" containerID="c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58" Jun 08 12:30:31.126671 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.126656 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58"} err="failed to get container status \"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58\": rpc error: code = NotFound desc = could not find container \"c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58\": container with ID starting with c96036ad88dd30bda75727d6a6b3503b8d76313f20510df5b46163d7df12bb58 not found: ID does not exist" Jun 08 12:30:31.126715 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.126672 2580 scope.go:117] "RemoveContainer" containerID="ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809" Jun 08 12:30:31.126857 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.126839 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809"} err="failed to get container status \"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809\": rpc error: code = NotFound desc = could not find container \"ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809\": container with ID starting with ec615443604cbea11ad424ae670655c72b9e626e6865f570d6b072faa5cb5809 not found: ID does not exist" Jun 08 12:30:31.126909 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.126856 2580 scope.go:117] "RemoveContainer" containerID="7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4" Jun 08 12:30:31.127100 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.127082 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4"} err="failed to get container status \"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4\": rpc error: code = NotFound desc = could not find container \"7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4\": container with ID starting with 7c0de501836bb0ed7c7dc7c22f9ba44fc86bd6fe361db8f414cf4bea313861c4 not found: ID does not exist" Jun 08 12:30:31.127151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.127101 2580 scope.go:117] "RemoveContainer" containerID="f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57" Jun 08 12:30:31.127347 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.127329 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57"} err="failed to get container status \"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57\": rpc error: code = NotFound desc = could not find container \"f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57\": container with ID starting with f55fd9f9bdf4ce1b3ba663bf3e6231a8eb462ee1e31a6e11380e6d927f7a0a57 not found: ID does not exist" Jun 08 12:30:31.127347 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.127346 2580 scope.go:117] "RemoveContainer" containerID="7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4" Jun 08 12:30:31.127547 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.127525 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4"} err="failed to get container status \"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4\": rpc error: code = NotFound desc = could not find container \"7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4\": container with ID starting with 7f5b280337ac88efb7ea1fbd86ceee91c9c11738de12fc8604481f5d93a331a4 not found: ID does not exist" Jun 08 12:30:31.859359 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:31.859329 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" path="/var/lib/kubelet/pods/df0c8d86-02d4-420e-a403-a7e59860ed5a/volumes" Jun 08 12:30:56.213580 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:30:56.213522 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:31:21.937473 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:21.937424 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:31:22.037307 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.037262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:31:22.226730 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.226696 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:31:22.323324 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323287 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw"] Jun 08 12:31:22.323601 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323587 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="offline" Jun 08 12:31:22.323649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323602 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="offline" Jun 08 12:31:22.323649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323612 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="ui" Jun 08 12:31:22.323649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323620 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="ui" Jun 08 12:31:22.323649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323633 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="feast-init" Jun 08 12:31:22.323649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323639 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="feast-init" Jun 08 12:31:22.323649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323644 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="registry" Jun 08 12:31:22.323649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323649 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="registry" Jun 08 12:31:22.323871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323658 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="online" Jun 08 12:31:22.323871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323663 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="online" Jun 08 12:31:22.323871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323672 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="feast-apply" Jun 08 12:31:22.323871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323676 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="feast-apply" Jun 08 12:31:22.323871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323726 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="offline" Jun 08 12:31:22.323871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323733 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="ui" Jun 08 12:31:22.323871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323742 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="online" Jun 08 12:31:22.323871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.323748 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="df0c8d86-02d4-420e-a403-a7e59860ed5a" containerName="registry" Jun 08 12:31:22.328216 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.328198 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.331416 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.331389 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jun 08 12:31:22.331521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.331399 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jun 08 12:31:22.331521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.331486 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jun 08 12:31:22.331703 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.331688 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jun 08 12:31:22.332500 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.332478 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-dtzqq\"" Jun 08 12:31:22.332605 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.332488 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jun 08 12:31:22.332605 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.332530 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jun 08 12:31:22.351838 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.351811 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw"] Jun 08 12:31:22.355183 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.355157 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:31:22.428935 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.428903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.428935 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.428955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debc3cb7-6809-4ec2-9496-53f152ac7ce3-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.429196 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.428987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcmx6\" (UniqueName: \"kubernetes.io/projected/debc3cb7-6809-4ec2-9496-53f152ac7ce3-kube-api-access-wcmx6\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.429196 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.429028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.429196 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.429099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.429196 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.429127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debc3cb7-6809-4ec2-9496-53f152ac7ce3-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.512525 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.512438 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:31:22.530351 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.530315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.530529 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.530360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debc3cb7-6809-4ec2-9496-53f152ac7ce3-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.530529 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.530383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcmx6\" (UniqueName: \"kubernetes.io/projected/debc3cb7-6809-4ec2-9496-53f152ac7ce3-kube-api-access-wcmx6\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.530529 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.530403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.530529 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.530431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.530529 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.530463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debc3cb7-6809-4ec2-9496-53f152ac7ce3-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.530903 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.530882 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debc3cb7-6809-4ec2-9496-53f152ac7ce3-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.531191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.531170 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debc3cb7-6809-4ec2-9496-53f152ac7ce3-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.532919 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.532896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.533056 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.532936 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.533056 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.533014 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.539478 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.539452 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcmx6\" (UniqueName: \"kubernetes.io/projected/debc3cb7-6809-4ec2-9496-53f152ac7ce3-kube-api-access-wcmx6\") pod \"feast-simple-feast-remote-setup-669bcf45fc-czhgw\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.609462 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.609427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:31:22.637572 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.637541 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:22.807594 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:22.807539 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw"] Jun 08 12:31:22.809835 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:31:22.809802 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddebc3cb7_6809_4ec2_9496_53f152ac7ce3.slice/crio-715cc96d5da6699c8757c056be777e876f46fca0aa2e76fc12d128c5ce747f91 WatchSource:0}: Error finding container 715cc96d5da6699c8757c056be777e876f46fca0aa2e76fc12d128c5ce747f91: Status 404 returned error can't find the container with id 715cc96d5da6699c8757c056be777e876f46fca0aa2e76fc12d128c5ce747f91 Jun 08 12:31:23.282596 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:23.282554 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerStarted","Data":"c65955cf9b90d3c75634fe2471944f5eb7b9882a87fc9a3a4d9b234a62967433"} Jun 08 12:31:23.283025 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:23.282608 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerStarted","Data":"715cc96d5da6699c8757c056be777e876f46fca0aa2e76fc12d128c5ce747f91"} Jun 08 12:31:28.302542 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:28.302493 2580 generic.go:356] "Generic (PLEG): container finished" podID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerID="c65955cf9b90d3c75634fe2471944f5eb7b9882a87fc9a3a4d9b234a62967433" exitCode=0 Jun 08 12:31:28.302914 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:28.302568 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerDied","Data":"c65955cf9b90d3c75634fe2471944f5eb7b9882a87fc9a3a4d9b234a62967433"} Jun 08 12:31:29.309454 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:29.309412 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerStarted","Data":"0c3575e5c7fc6b877d2bc9b9173414765f5936dba02d9a41bcbc53c8ae7c3f19"} Jun 08 12:31:34.331973 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:34.331922 2580 generic.go:356] "Generic (PLEG): container finished" podID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerID="0c3575e5c7fc6b877d2bc9b9173414765f5936dba02d9a41bcbc53c8ae7c3f19" exitCode=0 Jun 08 12:31:34.332353 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:34.331981 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerDied","Data":"0c3575e5c7fc6b877d2bc9b9173414765f5936dba02d9a41bcbc53c8ae7c3f19"} Jun 08 12:31:35.340765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:35.340734 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerStarted","Data":"4e01dff3d7418cdcb5bbc53a23aa725937d3f9c0daa031a02db80663f610d9a5"} Jun 08 12:31:35.341320 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:35.340777 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerStarted","Data":"01a5b0f259341dd1fce7df32a8803d62e6628f593d85b0cbbad09d23bcde2410"} Jun 08 12:31:35.341320 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:35.340789 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerStarted","Data":"f6f7948c359de5494810c81ed8980074e193613e306a99e510e5e5e4d270d231"} Jun 08 12:31:35.371707 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:35.371644 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podStartSLOduration=13.371622928 podStartE2EDuration="13.371622928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:31:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:31:35.369688903 +0000 UTC m=+1458.009530338" watchObservedRunningTime="2026-06-08 12:31:35.371622928 +0000 UTC m=+1458.011464365" Jun 08 12:31:37.637679 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:37.637641 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:37.638187 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:37.637704 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:37.638187 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:37.637720 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:37.639530 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:37.639492 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jun 08 12:31:37.639649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:37.639501 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jun 08 12:31:37.639649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:37.639498 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused" Jun 08 12:31:40.639091 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:40.639058 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:40.639584 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:40.639219 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:40.639665 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:40.639644 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:40.639724 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:40.639681 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:40.639911 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:40.639894 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:40.640037 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:40.640024 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:40.643478 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:40.643456 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:41.370032 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:41.370001 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:31:41.372916 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:31:41.372892 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:32:17.906298 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:17.906269 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:32:17.909276 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:17.909256 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:32:17.910378 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:17.910351 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:32:17.913290 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:17.913266 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:32:23.701291 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:23.701261 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:32:43.635848 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.635761 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw"] Jun 08 12:32:43.636311 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.635993 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="online" containerID="cri-o://f6f7948c359de5494810c81ed8980074e193613e306a99e510e5e5e4d270d231" gracePeriod=30 Jun 08 12:32:43.636311 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.636038 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="offline" containerID="cri-o://01a5b0f259341dd1fce7df32a8803d62e6628f593d85b0cbbad09d23bcde2410" gracePeriod=30 Jun 08 12:32:43.636311 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.636046 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="ui" containerID="cri-o://4e01dff3d7418cdcb5bbc53a23aa725937d3f9c0daa031a02db80663f610d9a5" gracePeriod=30 Jun 08 12:32:43.647876 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.647841 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:32:43.709530 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.709498 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:32:43.794203 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.794169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:32:43.794988 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.794969 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz"] Jun 08 12:32:43.795215 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.795185 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="registry" containerID="cri-o://5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03" gracePeriod=30 Jun 08 12:32:43.795352 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.795244 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="offline" containerID="cri-o://3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9" gracePeriod=30 Jun 08 12:32:43.795352 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.795253 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="ui" containerID="cri-o://00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113" gracePeriod=30 Jun 08 12:32:43.795468 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.795250 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="online" containerID="cri-o://d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6" gracePeriod=30 Jun 08 12:32:43.890011 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.889920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:32:43.990399 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:43.990360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:32:44.636656 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:44.636622 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerID="00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113" exitCode=0 Jun 08 12:32:44.636656 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:44.636647 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerID="d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6" exitCode=0 Jun 08 12:32:44.637092 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:44.636697 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerDied","Data":"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113"} Jun 08 12:32:44.637092 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:44.636742 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerDied","Data":"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6"} Jun 08 12:32:44.639343 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:44.639322 2580 generic.go:356] "Generic (PLEG): container finished" podID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerID="4e01dff3d7418cdcb5bbc53a23aa725937d3f9c0daa031a02db80663f610d9a5" exitCode=0 Jun 08 12:32:44.639343 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:44.639342 2580 generic.go:356] "Generic (PLEG): container finished" podID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerID="f6f7948c359de5494810c81ed8980074e193613e306a99e510e5e5e4d270d231" exitCode=0 Jun 08 12:32:44.639504 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:44.639393 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerDied","Data":"4e01dff3d7418cdcb5bbc53a23aa725937d3f9c0daa031a02db80663f610d9a5"} Jun 08 12:32:44.639504 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:44.639430 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerDied","Data":"f6f7948c359de5494810c81ed8980074e193613e306a99e510e5e5e4d270d231"} Jun 08 12:32:48.960235 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:48.960203 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:32:49.478598 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:49.478557 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Jun 08 12:32:50.035538 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:50.035487 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Jun 08 12:32:50.639798 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:50.639753 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jun 08 12:32:51.370300 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:51.370258 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jun 08 12:32:59.478897 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:32:59.478845 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Jun 08 12:33:00.035374 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:00.035324 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Jun 08 12:33:00.640622 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:00.640569 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jun 08 12:33:01.370511 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:01.370465 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jun 08 12:33:09.478132 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:09.478082 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Jun 08 12:33:09.478588 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:09.478210 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:33:10.035393 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:10.035349 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Jun 08 12:33:10.035772 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:10.035469 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:33:10.639929 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:10.639885 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jun 08 12:33:10.640382 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:10.640041 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:33:11.370589 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:11.370540 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jun 08 12:33:11.370801 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:11.370730 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:33:13.759796 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:13.759767 2580 generic.go:356] "Generic (PLEG): container finished" podID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerID="01a5b0f259341dd1fce7df32a8803d62e6628f593d85b0cbbad09d23bcde2410" exitCode=137 Jun 08 12:33:13.760127 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:13.759836 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerDied","Data":"01a5b0f259341dd1fce7df32a8803d62e6628f593d85b0cbbad09d23bcde2410"} Jun 08 12:33:14.282263 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.282243 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:33:14.352153 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352125 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-ui-tls\") pod \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " Jun 08 12:33:14.352330 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352162 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-online-tls\") pod \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " Jun 08 12:33:14.352330 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352194 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debc3cb7-6809-4ec2-9496-53f152ac7ce3-feast-data\") pod \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " Jun 08 12:33:14.352330 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352240 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debc3cb7-6809-4ec2-9496-53f152ac7ce3-registry-tls\") pod \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " Jun 08 12:33:14.352330 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352283 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcmx6\" (UniqueName: \"kubernetes.io/projected/debc3cb7-6809-4ec2-9496-53f152ac7ce3-kube-api-access-wcmx6\") pod \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " Jun 08 12:33:14.352617 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352334 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-offline-tls\") pod \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\" (UID: \"debc3cb7-6809-4ec2-9496-53f152ac7ce3\") " Jun 08 12:33:14.352680 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352624 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/debc3cb7-6809-4ec2-9496-53f152ac7ce3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "debc3cb7-6809-4ec2-9496-53f152ac7ce3" (UID: "debc3cb7-6809-4ec2-9496-53f152ac7ce3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:33:14.352817 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352772 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/debc3cb7-6809-4ec2-9496-53f152ac7ce3-feast-data" (OuterVolumeSpecName: "feast-data") pod "debc3cb7-6809-4ec2-9496-53f152ac7ce3" (UID: "debc3cb7-6809-4ec2-9496-53f152ac7ce3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:33:14.352884 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352856 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debc3cb7-6809-4ec2-9496-53f152ac7ce3-feast-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.352884 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.352873 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debc3cb7-6809-4ec2-9496-53f152ac7ce3-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.354475 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.354446 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-online-tls" (OuterVolumeSpecName: "online-tls") pod "debc3cb7-6809-4ec2-9496-53f152ac7ce3" (UID: "debc3cb7-6809-4ec2-9496-53f152ac7ce3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:33:14.354578 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.354485 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "debc3cb7-6809-4ec2-9496-53f152ac7ce3" (UID: "debc3cb7-6809-4ec2-9496-53f152ac7ce3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:33:14.355106 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.355081 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "debc3cb7-6809-4ec2-9496-53f152ac7ce3" (UID: "debc3cb7-6809-4ec2-9496-53f152ac7ce3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:33:14.355194 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.355114 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/debc3cb7-6809-4ec2-9496-53f152ac7ce3-kube-api-access-wcmx6" (OuterVolumeSpecName: "kube-api-access-wcmx6") pod "debc3cb7-6809-4ec2-9496-53f152ac7ce3" (UID: "debc3cb7-6809-4ec2-9496-53f152ac7ce3"). InnerVolumeSpecName "kube-api-access-wcmx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:33:14.431826 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.431803 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:33:14.453241 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.453220 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-ui-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.453241 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.453238 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-online-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.453375 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.453246 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcmx6\" (UniqueName: \"kubernetes.io/projected/debc3cb7-6809-4ec2-9496-53f152ac7ce3-kube-api-access-wcmx6\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.453375 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.453255 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debc3cb7-6809-4ec2-9496-53f152ac7ce3-offline-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.553656 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.553624 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-registry-tls\") pod \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " Jun 08 12:33:14.553798 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.553662 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwm6r\" (UniqueName: \"kubernetes.io/projected/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-kube-api-access-bwm6r\") pod \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " Jun 08 12:33:14.553798 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.553692 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls\") pod \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " Jun 08 12:33:14.553798 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.553727 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-ui-tls\") pod \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " Jun 08 12:33:14.553798 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.553781 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-feast-data\") pod \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " Jun 08 12:33:14.554110 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.553808 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls\") pod \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\" (UID: \"7bf36b67-bcac-486d-a8ae-270ea1dacc6d\") " Jun 08 12:33:14.554385 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.554348 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-feast-data" (OuterVolumeSpecName: "feast-data") pod "7bf36b67-bcac-486d-a8ae-270ea1dacc6d" (UID: "7bf36b67-bcac-486d-a8ae-270ea1dacc6d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:33:14.555848 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.555823 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls" (OuterVolumeSpecName: "online-tls") pod "7bf36b67-bcac-486d-a8ae-270ea1dacc6d" (UID: "7bf36b67-bcac-486d-a8ae-270ea1dacc6d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:33:14.556287 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.556264 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7bf36b67-bcac-486d-a8ae-270ea1dacc6d" (UID: "7bf36b67-bcac-486d-a8ae-270ea1dacc6d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:33:14.556375 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.556332 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-kube-api-access-bwm6r" (OuterVolumeSpecName: "kube-api-access-bwm6r") pod "7bf36b67-bcac-486d-a8ae-270ea1dacc6d" (UID: "7bf36b67-bcac-486d-a8ae-270ea1dacc6d"). InnerVolumeSpecName "kube-api-access-bwm6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:33:14.556375 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.556355 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7bf36b67-bcac-486d-a8ae-270ea1dacc6d" (UID: "7bf36b67-bcac-486d-a8ae-270ea1dacc6d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:33:14.556461 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.556369 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7bf36b67-bcac-486d-a8ae-270ea1dacc6d" (UID: "7bf36b67-bcac-486d-a8ae-270ea1dacc6d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:33:14.654863 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.654841 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.654863 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.654863 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwm6r\" (UniqueName: \"kubernetes.io/projected/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-kube-api-access-bwm6r\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.655006 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.654872 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-offline-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.655006 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.654881 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-ui-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.655006 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.654890 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-feast-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.655006 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.654897 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7bf36b67-bcac-486d-a8ae-270ea1dacc6d-online-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:33:14.766594 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.766569 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerID="3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9" exitCode=137 Jun 08 12:33:14.766594 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.766591 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerID="5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03" exitCode=137 Jun 08 12:33:14.767112 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.766654 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" Jun 08 12:33:14.767112 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.766662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerDied","Data":"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9"} Jun 08 12:33:14.767112 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.766713 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerDied","Data":"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03"} Jun 08 12:33:14.767112 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.766734 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz" event={"ID":"7bf36b67-bcac-486d-a8ae-270ea1dacc6d","Type":"ContainerDied","Data":"896bcb83fab5675b2cdf11839c3e6579f31b8f990d376f6550c6678905fb5328"} Jun 08 12:33:14.767112 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.766758 2580 scope.go:117] "RemoveContainer" containerID="00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113" Jun 08 12:33:14.769441 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.769420 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" event={"ID":"debc3cb7-6809-4ec2-9496-53f152ac7ce3","Type":"ContainerDied","Data":"715cc96d5da6699c8757c056be777e876f46fca0aa2e76fc12d128c5ce747f91"} Jun 08 12:33:14.769559 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.769487 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw" Jun 08 12:33:14.776848 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.776823 2580 scope.go:117] "RemoveContainer" containerID="3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9" Jun 08 12:33:14.784530 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.784513 2580 scope.go:117] "RemoveContainer" containerID="d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6" Jun 08 12:33:14.791996 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.791973 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz"] Jun 08 12:33:14.792671 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.792655 2580 scope.go:117] "RemoveContainer" containerID="5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03" Jun 08 12:33:14.801327 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.797641 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-g8qzz"] Jun 08 12:33:14.805182 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.805165 2580 scope.go:117] "RemoveContainer" containerID="a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202" Jun 08 12:33:14.811868 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.811850 2580 scope.go:117] "RemoveContainer" containerID="3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524" Jun 08 12:33:14.813547 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.813530 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw"] Jun 08 12:33:14.818268 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.818247 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-czhgw"] Jun 08 12:33:14.819652 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.819636 2580 scope.go:117] "RemoveContainer" containerID="00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113" Jun 08 12:33:14.819892 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:14.819873 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113\": container with ID starting with 00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113 not found: ID does not exist" containerID="00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113" Jun 08 12:33:14.819983 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.819898 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113"} err="failed to get container status \"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113\": rpc error: code = NotFound desc = could not find container \"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113\": container with ID starting with 00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113 not found: ID does not exist" Jun 08 12:33:14.819983 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.819914 2580 scope.go:117] "RemoveContainer" containerID="3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9" Jun 08 12:33:14.820160 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:14.820143 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9\": container with ID starting with 3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9 not found: ID does not exist" containerID="3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9" Jun 08 12:33:14.820200 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.820165 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9"} err="failed to get container status \"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9\": rpc error: code = NotFound desc = could not find container \"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9\": container with ID starting with 3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9 not found: ID does not exist" Jun 08 12:33:14.820200 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.820180 2580 scope.go:117] "RemoveContainer" containerID="d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6" Jun 08 12:33:14.820388 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:14.820371 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6\": container with ID starting with d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6 not found: ID does not exist" containerID="d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6" Jun 08 12:33:14.820430 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.820392 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6"} err="failed to get container status \"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6\": rpc error: code = NotFound desc = could not find container \"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6\": container with ID starting with d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6 not found: ID does not exist" Jun 08 12:33:14.820430 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.820405 2580 scope.go:117] "RemoveContainer" containerID="5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03" Jun 08 12:33:14.820594 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:14.820578 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03\": container with ID starting with 5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03 not found: ID does not exist" containerID="5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03" Jun 08 12:33:14.820630 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.820597 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03"} err="failed to get container status \"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03\": rpc error: code = NotFound desc = could not find container \"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03\": container with ID starting with 5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03 not found: ID does not exist" Jun 08 12:33:14.820630 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.820610 2580 scope.go:117] "RemoveContainer" containerID="a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202" Jun 08 12:33:14.820798 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:14.820784 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202\": container with ID starting with a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202 not found: ID does not exist" containerID="a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202" Jun 08 12:33:14.820839 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.820799 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202"} err="failed to get container status \"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202\": rpc error: code = NotFound desc = could not find container \"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202\": container with ID starting with a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202 not found: ID does not exist" Jun 08 12:33:14.820839 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.820810 2580 scope.go:117] "RemoveContainer" containerID="3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524" Jun 08 12:33:14.821025 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:14.821009 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524\": container with ID starting with 3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524 not found: ID does not exist" containerID="3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524" Jun 08 12:33:14.821077 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821027 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524"} err="failed to get container status \"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524\": rpc error: code = NotFound desc = could not find container \"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524\": container with ID starting with 3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524 not found: ID does not exist" Jun 08 12:33:14.821077 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821038 2580 scope.go:117] "RemoveContainer" containerID="00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113" Jun 08 12:33:14.821234 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821219 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113"} err="failed to get container status \"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113\": rpc error: code = NotFound desc = could not find container \"00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113\": container with ID starting with 00e9eafa759a36da3432a5dfaa4063eab7a68bae7a2281fc83eab6ef5edd3113 not found: ID does not exist" Jun 08 12:33:14.821273 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821235 2580 scope.go:117] "RemoveContainer" containerID="3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9" Jun 08 12:33:14.821411 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821397 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9"} err="failed to get container status \"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9\": rpc error: code = NotFound desc = could not find container \"3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9\": container with ID starting with 3c1761c3a911ccdfdff785e2491c3b39b42b8915016dd88486e2e61bc74582d9 not found: ID does not exist" Jun 08 12:33:14.821452 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821411 2580 scope.go:117] "RemoveContainer" containerID="d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6" Jun 08 12:33:14.821604 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821585 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6"} err="failed to get container status \"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6\": rpc error: code = NotFound desc = could not find container \"d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6\": container with ID starting with d274d114791039537d59417b3b2779289431318fcc22eb97d9cd1dabc21e84b6 not found: ID does not exist" Jun 08 12:33:14.821653 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821605 2580 scope.go:117] "RemoveContainer" containerID="5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03" Jun 08 12:33:14.821837 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821822 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03"} err="failed to get container status \"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03\": rpc error: code = NotFound desc = could not find container \"5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03\": container with ID starting with 5b06571798a5d884b7767c9bc106472b453822bc52ebb0b4c89f41ae2c4f7e03 not found: ID does not exist" Jun 08 12:33:14.821837 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.821836 2580 scope.go:117] "RemoveContainer" containerID="a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202" Jun 08 12:33:14.822068 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.822051 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202"} err="failed to get container status \"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202\": rpc error: code = NotFound desc = could not find container \"a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202\": container with ID starting with a956f05233d606477098a49979c18f2a7f76b67b09be3adcd52ef6513147e202 not found: ID does not exist" Jun 08 12:33:14.822114 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.822068 2580 scope.go:117] "RemoveContainer" containerID="3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524" Jun 08 12:33:14.822266 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.822251 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524"} err="failed to get container status \"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524\": rpc error: code = NotFound desc = could not find container \"3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524\": container with ID starting with 3a2dee9602a56cbab1c70cd498a90426ee8613264c3a25aa039ffd8524833524 not found: ID does not exist" Jun 08 12:33:14.822311 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.822267 2580 scope.go:117] "RemoveContainer" containerID="4e01dff3d7418cdcb5bbc53a23aa725937d3f9c0daa031a02db80663f610d9a5" Jun 08 12:33:14.829103 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.829090 2580 scope.go:117] "RemoveContainer" containerID="01a5b0f259341dd1fce7df32a8803d62e6628f593d85b0cbbad09d23bcde2410" Jun 08 12:33:14.835706 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.835691 2580 scope.go:117] "RemoveContainer" containerID="f6f7948c359de5494810c81ed8980074e193613e306a99e510e5e5e4d270d231" Jun 08 12:33:14.842532 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.842514 2580 scope.go:117] "RemoveContainer" containerID="0c3575e5c7fc6b877d2bc9b9173414765f5936dba02d9a41bcbc53c8ae7c3f19" Jun 08 12:33:14.849083 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:14.849067 2580 scope.go:117] "RemoveContainer" containerID="c65955cf9b90d3c75634fe2471944f5eb7b9882a87fc9a3a4d9b234a62967433" Jun 08 12:33:15.860289 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:15.860258 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" path="/var/lib/kubelet/pods/7bf36b67-bcac-486d-a8ae-270ea1dacc6d/volumes" Jun 08 12:33:15.860845 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:15.860832 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" path="/var/lib/kubelet/pods/debc3cb7-6809-4ec2-9496-53f152ac7ce3/volumes" Jun 08 12:33:26.732423 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:26.732388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:28.108820 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:28.108790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:30.572758 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:30.572708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:32.563275 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:32.563232 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:32.631232 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:32.631190 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:53.910373 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:53.910306 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:54.009301 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:54.009258 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:55.617566 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.617525 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:55.720153 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.720122 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:55.822029 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.821991 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw"] Jun 08 12:33:55.822280 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822268 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="feast-init" Jun 08 12:33:55.822329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822282 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="feast-init" Jun 08 12:33:55.822329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822292 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="feast-apply" Jun 08 12:33:55.822329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822298 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="feast-apply" Jun 08 12:33:55.822329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822306 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="feast-init" Jun 08 12:33:55.822329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822314 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="feast-init" Jun 08 12:33:55.822329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822319 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="feast-apply" Jun 08 12:33:55.822329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822324 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="feast-apply" Jun 08 12:33:55.822329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822331 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="registry" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822336 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="registry" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822347 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="offline" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822353 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="offline" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822360 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="online" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822365 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="online" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822373 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="ui" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822378 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="ui" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822389 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="online" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822394 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="online" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822399 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="offline" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822403 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="offline" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822407 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="ui" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822412 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="ui" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822458 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="offline" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822466 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="online" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822471 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="ui" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822477 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="ui" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822482 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="offline" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822488 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="debc3cb7-6809-4ec2-9496-53f152ac7ce3" containerName="online" Jun 08 12:33:55.822570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.822494 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf36b67-bcac-486d-a8ae-270ea1dacc6d" containerName="registry" Jun 08 12:33:55.825356 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.825332 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:55.825483 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.825466 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.829396 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.829378 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jun 08 12:33:55.829654 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.829634 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-n7rwg\"" Jun 08 12:33:55.829742 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.829727 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jun 08 12:33:55.829880 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.829865 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jun 08 12:33:55.830161 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.830145 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jun 08 12:33:55.830242 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.830181 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 08 12:33:55.830242 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.830190 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 08 12:33:55.838880 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.838853 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw"] Jun 08 12:33:55.887953 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.887868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6xmz\" (UniqueName: \"kubernetes.io/projected/1d4505e0-700a-4b76-bb3d-b760af94efd8-kube-api-access-k6xmz\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.887953 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.887899 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4505e0-700a-4b76-bb3d-b760af94efd8-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.887953 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.887923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.888186 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.888017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.888186 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.888043 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.888186 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.888061 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.928074 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.928044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:55.988864 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.988830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.988864 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.988869 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.989191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.988889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.989191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.988935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xmz\" (UniqueName: \"kubernetes.io/projected/1d4505e0-700a-4b76-bb3d-b760af94efd8-kube-api-access-k6xmz\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.989191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.989054 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4505e0-700a-4b76-bb3d-b760af94efd8-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.989191 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:55.989061 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jun 08 12:33:55.989191 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.989102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.989191 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:55.989152 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls podName:1d4505e0-700a-4b76-bb3d-b760af94efd8 nodeName:}" failed. No retries permitted until 2026-06-08 12:33:56.48912595 +0000 UTC m=+1599.128967365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls") pod "feast-simple-feast-setup-75c64c888b-lfsbw" (UID: "1d4505e0-700a-4b76-bb3d-b760af94efd8") : secret "feast-simple-feast-setup-registry-tls" not found Jun 08 12:33:55.989191 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:55.989182 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jun 08 12:33:55.989575 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:33:55.989227 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls podName:1d4505e0-700a-4b76-bb3d-b760af94efd8 nodeName:}" failed. No retries permitted until 2026-06-08 12:33:56.489213201 +0000 UTC m=+1599.129054617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls") pod "feast-simple-feast-setup-75c64c888b-lfsbw" (UID: "1d4505e0-700a-4b76-bb3d-b760af94efd8") : secret "feast-simple-feast-setup-online-tls" not found Jun 08 12:33:55.989575 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.989522 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4505e0-700a-4b76-bb3d-b760af94efd8-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.991329 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.991299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.991455 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.991372 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:55.999863 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:55.999834 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6xmz\" (UniqueName: \"kubernetes.io/projected/1d4505e0-700a-4b76-bb3d-b760af94efd8-kube-api-access-k6xmz\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:56.091528 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.091494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:56.242466 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.242437 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:33:56.494551 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.494463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:56.494551 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.494522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:56.496909 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.496879 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:56.497047 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.496959 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-lfsbw\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:56.734733 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.734697 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:33:56.869012 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.868984 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw"] Jun 08 12:33:56.870016 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:33:56.869925 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d4505e0_700a_4b76_bb3d_b760af94efd8.slice/crio-a34a7989c7ae65d8151460b418b9cc3989a42624409c4fdf75c7aa86faddadec WatchSource:0}: Error finding container a34a7989c7ae65d8151460b418b9cc3989a42624409c4fdf75c7aa86faddadec: Status 404 returned error can't find the container with id a34a7989c7ae65d8151460b418b9cc3989a42624409c4fdf75c7aa86faddadec Jun 08 12:33:56.925086 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.925060 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerStarted","Data":"bb6fd35f9897a0307c5be5fa55795a17ab6f99b5ce8ccdda8e707b95f7f99dea"} Jun 08 12:33:56.925222 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:33:56.925113 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerStarted","Data":"a34a7989c7ae65d8151460b418b9cc3989a42624409c4fdf75c7aa86faddadec"} Jun 08 12:34:01.946051 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:01.946019 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerID="bb6fd35f9897a0307c5be5fa55795a17ab6f99b5ce8ccdda8e707b95f7f99dea" exitCode=0 Jun 08 12:34:01.946494 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:01.946091 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerDied","Data":"bb6fd35f9897a0307c5be5fa55795a17ab6f99b5ce8ccdda8e707b95f7f99dea"} Jun 08 12:34:02.952538 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:02.952504 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerStarted","Data":"0b390e5188972ad66c5dda3d8cc447ca2dedddafbf62929d79f650365cbb0073"} Jun 08 12:34:07.974388 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:07.974351 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerID="0b390e5188972ad66c5dda3d8cc447ca2dedddafbf62929d79f650365cbb0073" exitCode=0 Jun 08 12:34:07.974752 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:07.974392 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerDied","Data":"0b390e5188972ad66c5dda3d8cc447ca2dedddafbf62929d79f650365cbb0073"} Jun 08 12:34:08.990320 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:08.990279 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerStarted","Data":"8c1c425c00e8ad1b3b0485c09a37a69aa80d6c57a7084bc323a87f3e0c034dba"} Jun 08 12:34:08.990812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:08.990344 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerStarted","Data":"6e2e1fa2a85f009e01bd6192e4d2e67701dda5a65d84f5ddae7ea055afa8f4a0"} Jun 08 12:34:08.990812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:08.990364 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerStarted","Data":"b4d1896f4b2816e4120b5058e357c3d03ede9a8d358864a42c69e298470dfcaf"} Jun 08 12:34:08.990812 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:08.990382 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerStarted","Data":"6a5be0db9758f1ff7d9f5e10493028342b589f3628e0ff12aab4348134e63dfc"} Jun 08 12:34:09.022107 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:09.022042 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podStartSLOduration=14.022017345 podStartE2EDuration="14.022017345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:34:09.018884608 +0000 UTC m=+1611.658726043" watchObservedRunningTime="2026-06-08 12:34:09.022017345 +0000 UTC m=+1611.661858780" Jun 08 12:34:11.735894 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:11.735853 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:11.736464 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:11.736438 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:11.736607 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:11.736472 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:11.736607 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:11.736490 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:11.737895 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:11.737857 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Jun 08 12:34:11.737895 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:11.737881 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.23:6571: connect: connection refused" Jun 08 12:34:11.738101 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:11.737889 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.23:8816: connect: connection refused" Jun 08 12:34:11.738101 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:11.738010 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Jun 08 12:34:14.736493 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.736459 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.736974 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.736610 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.736974 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.736840 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.737189 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.737164 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.737315 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.737212 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.737315 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.737232 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.737518 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.737497 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.737639 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.737624 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.737685 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.737680 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:14.741151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:14.741135 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:15.020137 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:15.020049 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:34:15.023087 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:34:15.023062 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:35:17.367033 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.366984 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw"] Jun 08 12:35:17.367536 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.367295 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="registry" containerID="cri-o://6a5be0db9758f1ff7d9f5e10493028342b589f3628e0ff12aab4348134e63dfc" gracePeriod=30 Jun 08 12:35:17.367536 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.367330 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="ui" containerID="cri-o://8c1c425c00e8ad1b3b0485c09a37a69aa80d6c57a7084bc323a87f3e0c034dba" gracePeriod=30 Jun 08 12:35:17.367536 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.367346 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="offline" containerID="cri-o://6e2e1fa2a85f009e01bd6192e4d2e67701dda5a65d84f5ddae7ea055afa8f4a0" gracePeriod=30 Jun 08 12:35:17.367536 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.367382 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="online" containerID="cri-o://b4d1896f4b2816e4120b5058e357c3d03ede9a8d358864a42c69e298470dfcaf" gracePeriod=30 Jun 08 12:35:17.370243 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.370206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:17.444822 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.444794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:17.556390 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.556353 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:17.730594 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.730562 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:17.818612 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.818583 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:17.868954 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.868915 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg"] Jun 08 12:35:17.874937 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.874903 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:17.881026 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.880998 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-7vqtn\"" Jun 08 12:35:17.892875 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.892838 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg"] Jun 08 12:35:17.897785 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.897753 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:17.964696 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.964654 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:17.967759 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.967736 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2eb3a7-0253-41b0-99a1-81d4f8176548-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:17.967829 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.967780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:17.967829 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.967807 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:17.967898 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.967830 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7klj\" (UniqueName: \"kubernetes.io/projected/fe2eb3a7-0253-41b0-99a1-81d4f8176548-kube-api-access-l7klj\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:17.967898 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.967874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:17.967995 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:17.967926 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.068821 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.068725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.069008 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.068882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.069008 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.068923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2eb3a7-0253-41b0-99a1-81d4f8176548-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.069136 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.069005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.069136 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.069053 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.069136 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.069074 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7klj\" (UniqueName: \"kubernetes.io/projected/fe2eb3a7-0253-41b0-99a1-81d4f8176548-kube-api-access-l7klj\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.069398 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.069368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2eb3a7-0253-41b0-99a1-81d4f8176548-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.071644 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.071614 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.071788 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.071664 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.071788 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.071668 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.071788 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.071759 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.085315 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.085293 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7klj\" (UniqueName: \"kubernetes.io/projected/fe2eb3a7-0253-41b0-99a1-81d4f8176548-kube-api-access-l7klj\") pod \"feast-simple-feast-setup-75c64c888b-47gtg\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.184669 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.184649 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:18.264752 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.264719 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerID="8c1c425c00e8ad1b3b0485c09a37a69aa80d6c57a7084bc323a87f3e0c034dba" exitCode=0 Jun 08 12:35:18.264752 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.264749 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerID="b4d1896f4b2816e4120b5058e357c3d03ede9a8d358864a42c69e298470dfcaf" exitCode=0 Jun 08 12:35:18.264989 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.264845 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerDied","Data":"8c1c425c00e8ad1b3b0485c09a37a69aa80d6c57a7084bc323a87f3e0c034dba"} Jun 08 12:35:18.264989 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.264885 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerDied","Data":"b4d1896f4b2816e4120b5058e357c3d03ede9a8d358864a42c69e298470dfcaf"} Jun 08 12:35:18.323402 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:18.323376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg"] Jun 08 12:35:18.324785 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:35:18.324758 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe2eb3a7_0253_41b0_99a1_81d4f8176548.slice/crio-8c1b8e9050aac8b08a215966e4e4ba48a0a3682a4a27c554ec91c896071ef666 WatchSource:0}: Error finding container 8c1b8e9050aac8b08a215966e4e4ba48a0a3682a4a27c554ec91c896071ef666: Status 404 returned error can't find the container with id 8c1b8e9050aac8b08a215966e4e4ba48a0a3682a4a27c554ec91c896071ef666 Jun 08 12:35:19.270056 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:19.270009 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerStarted","Data":"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61"} Jun 08 12:35:19.270056 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:19.270065 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerStarted","Data":"8c1b8e9050aac8b08a215966e4e4ba48a0a3682a4a27c554ec91c896071ef666"} Jun 08 12:35:23.290510 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:23.290477 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerID="68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61" exitCode=0 Jun 08 12:35:23.290935 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:23.290532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerDied","Data":"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61"} Jun 08 12:35:24.296868 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:24.296833 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerStarted","Data":"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e"} Jun 08 12:35:24.737430 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:24.737376 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Jun 08 12:35:25.020367 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:25.020280 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Jun 08 12:35:26.891962 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:26.891906 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:26.959209 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:26.959174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:35:29.318855 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:29.318819 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerID="e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e" exitCode=0 Jun 08 12:35:29.319252 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:29.318890 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerDied","Data":"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e"} Jun 08 12:35:30.333567 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:30.333518 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerStarted","Data":"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204"} Jun 08 12:35:30.334102 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:30.333576 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerStarted","Data":"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576"} Jun 08 12:35:30.334102 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:30.333597 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerStarted","Data":"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379"} Jun 08 12:35:30.334102 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:30.333614 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerStarted","Data":"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52"} Jun 08 12:35:30.371185 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:30.371111 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podStartSLOduration=13.371089922 podStartE2EDuration="13.371089922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:35:30.365722873 +0000 UTC m=+1693.005564348" watchObservedRunningTime="2026-06-08 12:35:30.371089922 +0000 UTC m=+1693.010931357" Jun 08 12:35:33.185111 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:33.185074 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:33.185605 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:33.185137 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:33.185605 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:33.185159 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:33.185750 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:33.185725 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:33.186749 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:33.186708 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jun 08 12:35:33.186873 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:33.186752 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused" Jun 08 12:35:33.186873 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:33.186772 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Jun 08 12:35:33.186873 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:33.186847 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jun 08 12:35:34.738136 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:34.738093 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Jun 08 12:35:35.020614 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:35.020519 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Jun 08 12:35:36.186290 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.186211 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.186687 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.186455 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.186909 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.186890 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.187017 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.186929 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.187017 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.186987 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.190855 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.190835 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.366360 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.366328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.366545 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.366371 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.366545 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.366390 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.366754 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.366738 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:36.369049 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:36.369031 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:37.370551 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:37.370527 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:35:44.738257 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:44.738210 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Jun 08 12:35:44.738717 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:44.738343 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:35:45.021235 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:45.021148 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Jun 08 12:35:45.021508 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:45.021265 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:35:47.411090 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:47.411056 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerID="6e2e1fa2a85f009e01bd6192e4d2e67701dda5a65d84f5ddae7ea055afa8f4a0" exitCode=137 Jun 08 12:35:47.411090 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:47.411084 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerID="6a5be0db9758f1ff7d9f5e10493028342b589f3628e0ff12aab4348134e63dfc" exitCode=137 Jun 08 12:35:47.411435 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:47.411136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerDied","Data":"6e2e1fa2a85f009e01bd6192e4d2e67701dda5a65d84f5ddae7ea055afa8f4a0"} Jun 08 12:35:47.411435 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:47.411190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerDied","Data":"6a5be0db9758f1ff7d9f5e10493028342b589f3628e0ff12aab4348134e63dfc"} Jun 08 12:35:48.019803 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.019780 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:35:48.134222 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.134135 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls\") pod \"1d4505e0-700a-4b76-bb3d-b760af94efd8\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " Jun 08 12:35:48.134222 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.134172 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls\") pod \"1d4505e0-700a-4b76-bb3d-b760af94efd8\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " Jun 08 12:35:48.134446 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.134230 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4505e0-700a-4b76-bb3d-b760af94efd8-feast-data\") pod \"1d4505e0-700a-4b76-bb3d-b760af94efd8\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " Jun 08 12:35:48.134446 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.134272 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-ui-tls\") pod \"1d4505e0-700a-4b76-bb3d-b760af94efd8\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " Jun 08 12:35:48.134446 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.134323 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6xmz\" (UniqueName: \"kubernetes.io/projected/1d4505e0-700a-4b76-bb3d-b760af94efd8-kube-api-access-k6xmz\") pod \"1d4505e0-700a-4b76-bb3d-b760af94efd8\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " Jun 08 12:35:48.134446 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.134356 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-offline-tls\") pod \"1d4505e0-700a-4b76-bb3d-b760af94efd8\" (UID: \"1d4505e0-700a-4b76-bb3d-b760af94efd8\") " Jun 08 12:35:48.134782 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.134751 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4505e0-700a-4b76-bb3d-b760af94efd8-feast-data" (OuterVolumeSpecName: "feast-data") pod "1d4505e0-700a-4b76-bb3d-b760af94efd8" (UID: "1d4505e0-700a-4b76-bb3d-b760af94efd8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:35:48.136634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.136603 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "1d4505e0-700a-4b76-bb3d-b760af94efd8" (UID: "1d4505e0-700a-4b76-bb3d-b760af94efd8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:35:48.136886 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.136854 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1d4505e0-700a-4b76-bb3d-b760af94efd8" (UID: "1d4505e0-700a-4b76-bb3d-b760af94efd8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:35:48.136985 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.136879 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls" (OuterVolumeSpecName: "online-tls") pod "1d4505e0-700a-4b76-bb3d-b760af94efd8" (UID: "1d4505e0-700a-4b76-bb3d-b760af94efd8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:35:48.136985 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.136911 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "1d4505e0-700a-4b76-bb3d-b760af94efd8" (UID: "1d4505e0-700a-4b76-bb3d-b760af94efd8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:35:48.136985 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.136923 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4505e0-700a-4b76-bb3d-b760af94efd8-kube-api-access-k6xmz" (OuterVolumeSpecName: "kube-api-access-k6xmz") pod "1d4505e0-700a-4b76-bb3d-b760af94efd8" (UID: "1d4505e0-700a-4b76-bb3d-b760af94efd8"). InnerVolumeSpecName "kube-api-access-k6xmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:35:48.235326 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.235300 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4505e0-700a-4b76-bb3d-b760af94efd8-feast-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:35:48.235326 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.235325 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-ui-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:35:48.235474 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.235335 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6xmz\" (UniqueName: \"kubernetes.io/projected/1d4505e0-700a-4b76-bb3d-b760af94efd8-kube-api-access-k6xmz\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:35:48.235474 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.235345 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-offline-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:35:48.235474 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.235354 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-online-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:35:48.235474 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.235361 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1d4505e0-700a-4b76-bb3d-b760af94efd8-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:35:48.417174 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.417103 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" event={"ID":"1d4505e0-700a-4b76-bb3d-b760af94efd8","Type":"ContainerDied","Data":"a34a7989c7ae65d8151460b418b9cc3989a42624409c4fdf75c7aa86faddadec"} Jun 08 12:35:48.417174 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.417161 2580 scope.go:117] "RemoveContainer" containerID="8c1c425c00e8ad1b3b0485c09a37a69aa80d6c57a7084bc323a87f3e0c034dba" Jun 08 12:35:48.417174 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.417168 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw" Jun 08 12:35:48.426241 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.426224 2580 scope.go:117] "RemoveContainer" containerID="6e2e1fa2a85f009e01bd6192e4d2e67701dda5a65d84f5ddae7ea055afa8f4a0" Jun 08 12:35:48.433439 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.433423 2580 scope.go:117] "RemoveContainer" containerID="b4d1896f4b2816e4120b5058e357c3d03ede9a8d358864a42c69e298470dfcaf" Jun 08 12:35:48.440743 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.440725 2580 scope.go:117] "RemoveContainer" containerID="6a5be0db9758f1ff7d9f5e10493028342b589f3628e0ff12aab4348134e63dfc" Jun 08 12:35:48.440880 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.440860 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw"] Jun 08 12:35:48.445042 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.445020 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lfsbw"] Jun 08 12:35:48.448196 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.448181 2580 scope.go:117] "RemoveContainer" containerID="0b390e5188972ad66c5dda3d8cc447ca2dedddafbf62929d79f650365cbb0073" Jun 08 12:35:48.456866 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:48.456851 2580 scope.go:117] "RemoveContainer" containerID="bb6fd35f9897a0307c5be5fa55795a17ab6f99b5ce8ccdda8e707b95f7f99dea" Jun 08 12:35:49.859515 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:35:49.859482 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" path="/var/lib/kubelet/pods/1d4505e0-700a-4b76-bb3d-b760af94efd8/volumes" Jun 08 12:36:28.339464 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:28.339423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:36:38.725504 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:38.725470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:36:38.848819 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:38.848782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:36:38.977170 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:38.977077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:36:39.080519 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.080480 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:36:39.197810 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.197768 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj"] Jun 08 12:36:39.198512 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198488 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="offline" Jun 08 12:36:39.198648 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198611 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="offline" Jun 08 12:36:39.198648 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198644 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="registry" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198652 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="registry" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198680 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="feast-init" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198689 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="feast-init" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198699 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="feast-apply" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198708 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="feast-apply" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198717 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="ui" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198726 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="ui" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198738 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="online" Jun 08 12:36:39.198765 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198748 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="online" Jun 08 12:36:39.199249 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198831 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="online" Jun 08 12:36:39.199249 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198842 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="ui" Jun 08 12:36:39.199249 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198854 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="offline" Jun 08 12:36:39.199249 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.198862 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4505e0-700a-4b76-bb3d-b760af94efd8" containerName="registry" Jun 08 12:36:39.203385 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.203365 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.212816 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.212796 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jun 08 12:36:39.212928 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.212858 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jun 08 12:36:39.212928 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.212873 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jun 08 12:36:39.213074 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.212968 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jun 08 12:36:39.213275 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.213259 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-s7ckd\"" Jun 08 12:36:39.213654 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.213631 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jun 08 12:36:39.213746 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.213696 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jun 08 12:36:39.246150 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.246065 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj"] Jun 08 12:36:39.247606 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.247574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.247739 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.247615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/619a7b38-ba41-4dfd-b11a-bbcf1785d977-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.247739 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.247720 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/619a7b38-ba41-4dfd-b11a-bbcf1785d977-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.247853 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.247759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.247853 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.247785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqmwk\" (UniqueName: \"kubernetes.io/projected/619a7b38-ba41-4dfd-b11a-bbcf1785d977-kube-api-access-tqmwk\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.247974 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.247894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.349252 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.349189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.349464 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.349263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/619a7b38-ba41-4dfd-b11a-bbcf1785d977-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.349464 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.349325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/619a7b38-ba41-4dfd-b11a-bbcf1785d977-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.349464 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.349351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.349464 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.349375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqmwk\" (UniqueName: \"kubernetes.io/projected/619a7b38-ba41-4dfd-b11a-bbcf1785d977-kube-api-access-tqmwk\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.349464 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.349415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.349757 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:36:39.349509 2580 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Jun 08 12:36:39.349757 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:36:39.349598 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls podName:619a7b38-ba41-4dfd-b11a-bbcf1785d977 nodeName:}" failed. No retries permitted until 2026-06-08 12:36:39.849575689 +0000 UTC m=+1762.489417103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls") pod "feast-simple-feast-remote-setup-669bcf45fc-b6qbj" (UID: "619a7b38-ba41-4dfd-b11a-bbcf1785d977") : secret "feast-simple-feast-remote-setup-online-tls" not found Jun 08 12:36:39.349757 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:36:39.349619 2580 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Jun 08 12:36:39.349757 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:36:39.349675 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls podName:619a7b38-ba41-4dfd-b11a-bbcf1785d977 nodeName:}" failed. No retries permitted until 2026-06-08 12:36:39.84965651 +0000 UTC m=+1762.489497938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls") pod "feast-simple-feast-remote-setup-669bcf45fc-b6qbj" (UID: "619a7b38-ba41-4dfd-b11a-bbcf1785d977") : secret "feast-simple-feast-remote-setup-ui-tls" not found Jun 08 12:36:39.349757 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.349740 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/619a7b38-ba41-4dfd-b11a-bbcf1785d977-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.350332 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.350310 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/619a7b38-ba41-4dfd-b11a-bbcf1785d977-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.352271 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.352243 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.372888 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.372855 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqmwk\" (UniqueName: \"kubernetes.io/projected/619a7b38-ba41-4dfd-b11a-bbcf1785d977-kube-api-access-tqmwk\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.454580 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.454547 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:36:39.782056 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.782013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:36:39.854629 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.854593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.854790 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.854690 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.857224 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.857196 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:39.857355 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:39.857196 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-b6qbj\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:40.112233 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:40.112131 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:40.243980 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:40.243927 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj"] Jun 08 12:36:40.246264 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:36:40.246236 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod619a7b38_ba41_4dfd_b11a_bbcf1785d977.slice/crio-5f3e9b1f29a49a08da7ff2348fb273b05e7cdda1f4247266d0352e092d6a6bd1 WatchSource:0}: Error finding container 5f3e9b1f29a49a08da7ff2348fb273b05e7cdda1f4247266d0352e092d6a6bd1: Status 404 returned error can't find the container with id 5f3e9b1f29a49a08da7ff2348fb273b05e7cdda1f4247266d0352e092d6a6bd1 Jun 08 12:36:40.615415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:40.615377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerStarted","Data":"ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8"} Jun 08 12:36:40.615634 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:40.615424 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerStarted","Data":"5f3e9b1f29a49a08da7ff2348fb273b05e7cdda1f4247266d0352e092d6a6bd1"} Jun 08 12:36:45.637489 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:45.637454 2580 generic.go:356] "Generic (PLEG): container finished" podID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerID="ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8" exitCode=0 Jun 08 12:36:45.637871 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:45.637532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerDied","Data":"ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8"} Jun 08 12:36:46.642916 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:46.642882 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerStarted","Data":"44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f"} Jun 08 12:36:51.665475 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:51.665382 2580 generic.go:356] "Generic (PLEG): container finished" podID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerID="44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f" exitCode=0 Jun 08 12:36:51.665475 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:51.665449 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerDied","Data":"44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f"} Jun 08 12:36:52.674579 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:52.674534 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerStarted","Data":"3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794"} Jun 08 12:36:52.675136 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:52.674602 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerStarted","Data":"7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b"} Jun 08 12:36:52.675136 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:52.674622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerStarted","Data":"aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6"} Jun 08 12:36:52.707849 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:52.707783 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podStartSLOduration=13.707759389 podStartE2EDuration="13.707759389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:36:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:36:52.707062081 +0000 UTC m=+1775.346903515" watchObservedRunningTime="2026-06-08 12:36:52.707759389 +0000 UTC m=+1775.347600821" Jun 08 12:36:55.112823 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:55.112779 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:55.113363 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:55.112852 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:55.113363 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:55.112875 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:55.114570 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:55.114532 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused" Jun 08 12:36:55.114734 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:55.114571 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Jun 08 12:36:55.114734 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:55.114532 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Jun 08 12:36:58.113252 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.113215 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:58.113618 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.113318 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:58.113771 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.113729 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:58.113839 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.113777 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:58.113932 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.113915 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:58.114109 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.114089 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:58.117515 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.117497 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:58.702906 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.702872 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:36:58.705760 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:36:58.705740 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:37:17.932230 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:37:17.932201 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:37:17.936501 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:37:17.936477 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:37:17.937202 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:37:17.937183 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:37:17.941338 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:37:17.941322 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:37:55.231627 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:37:55.231590 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:00.093768 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.093737 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj"] Jun 08 12:38:00.096238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.093841 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:00.096238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.094036 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="online" containerID="cri-o://aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6" gracePeriod=30 Jun 08 12:38:00.096238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.094061 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="ui" containerID="cri-o://3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794" gracePeriod=30 Jun 08 12:38:00.096238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.094080 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="offline" containerID="cri-o://7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b" gracePeriod=30 Jun 08 12:38:00.192739 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.192687 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:00.222637 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.222605 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg"] Jun 08 12:38:00.222853 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.222823 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="registry" containerID="cri-o://1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52" gracePeriod=30 Jun 08 12:38:00.222926 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.222865 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="ui" containerID="cri-o://74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204" gracePeriod=30 Jun 08 12:38:00.223001 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.222917 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="online" containerID="cri-o://73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379" gracePeriod=30 Jun 08 12:38:00.223060 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.222916 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="offline" containerID="cri-o://c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576" gracePeriod=30 Jun 08 12:38:00.291606 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.291560 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:00.394641 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.394556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:00.952165 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.952139 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerID="74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204" exitCode=0 Jun 08 12:38:00.952165 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.952162 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerID="73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379" exitCode=0 Jun 08 12:38:00.952328 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.952192 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerDied","Data":"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204"} Jun 08 12:38:00.952328 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.952237 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerDied","Data":"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379"} Jun 08 12:38:00.954662 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.954640 2580 generic.go:356] "Generic (PLEG): container finished" podID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerID="3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794" exitCode=0 Jun 08 12:38:00.954662 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.954661 2580 generic.go:356] "Generic (PLEG): container finished" podID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerID="aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6" exitCode=0 Jun 08 12:38:00.954807 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.954711 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerDied","Data":"3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794"} Jun 08 12:38:00.954807 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:00.954745 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerDied","Data":"aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6"} Jun 08 12:38:05.394079 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:05.394043 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:06.187281 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:06.187242 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jun 08 12:38:06.366527 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:06.366481 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jun 08 12:38:08.113819 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:08.113779 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Jun 08 12:38:08.703262 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:08.703221 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Jun 08 12:38:16.186610 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:16.186575 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jun 08 12:38:16.366959 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:16.366899 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jun 08 12:38:18.114279 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:18.114239 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Jun 08 12:38:18.703038 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:18.702996 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Jun 08 12:38:26.186677 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:26.186633 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jun 08 12:38:26.187103 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:26.186761 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:38:26.367112 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:26.367070 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jun 08 12:38:26.367359 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:26.367176 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:38:28.113895 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:28.113851 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Jun 08 12:38:28.114375 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:28.114016 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:38:28.703704 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:28.703663 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Jun 08 12:38:28.703879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:28.703786 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:38:30.305297 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.305273 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:38:30.409601 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.409532 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-offline-tls\") pod \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " Jun 08 12:38:30.409601 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.409575 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls\") pod \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " Jun 08 12:38:30.409810 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.409618 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/619a7b38-ba41-4dfd-b11a-bbcf1785d977-feast-data\") pod \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " Jun 08 12:38:30.409810 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.409655 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls\") pod \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " Jun 08 12:38:30.409810 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.409683 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/619a7b38-ba41-4dfd-b11a-bbcf1785d977-registry-tls\") pod \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " Jun 08 12:38:30.409810 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.409755 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqmwk\" (UniqueName: \"kubernetes.io/projected/619a7b38-ba41-4dfd-b11a-bbcf1785d977-kube-api-access-tqmwk\") pod \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\" (UID: \"619a7b38-ba41-4dfd-b11a-bbcf1785d977\") " Jun 08 12:38:30.410177 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.410127 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/619a7b38-ba41-4dfd-b11a-bbcf1785d977-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "619a7b38-ba41-4dfd-b11a-bbcf1785d977" (UID: "619a7b38-ba41-4dfd-b11a-bbcf1785d977"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:38:30.410309 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.410193 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/619a7b38-ba41-4dfd-b11a-bbcf1785d977-feast-data" (OuterVolumeSpecName: "feast-data") pod "619a7b38-ba41-4dfd-b11a-bbcf1785d977" (UID: "619a7b38-ba41-4dfd-b11a-bbcf1785d977"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:38:30.411895 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.411869 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "619a7b38-ba41-4dfd-b11a-bbcf1785d977" (UID: "619a7b38-ba41-4dfd-b11a-bbcf1785d977"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:38:30.412006 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.411886 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/619a7b38-ba41-4dfd-b11a-bbcf1785d977-kube-api-access-tqmwk" (OuterVolumeSpecName: "kube-api-access-tqmwk") pod "619a7b38-ba41-4dfd-b11a-bbcf1785d977" (UID: "619a7b38-ba41-4dfd-b11a-bbcf1785d977"). InnerVolumeSpecName "kube-api-access-tqmwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:38:30.412006 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.411914 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls" (OuterVolumeSpecName: "online-tls") pod "619a7b38-ba41-4dfd-b11a-bbcf1785d977" (UID: "619a7b38-ba41-4dfd-b11a-bbcf1785d977"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:38:30.412006 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.411938 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "619a7b38-ba41-4dfd-b11a-bbcf1785d977" (UID: "619a7b38-ba41-4dfd-b11a-bbcf1785d977"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:38:30.510961 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.510913 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-offline-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:30.510961 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.510935 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-ui-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:30.510961 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.510958 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/619a7b38-ba41-4dfd-b11a-bbcf1785d977-feast-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:30.511151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.510969 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/619a7b38-ba41-4dfd-b11a-bbcf1785d977-online-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:30.511151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.510982 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/619a7b38-ba41-4dfd-b11a-bbcf1785d977-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:30.511151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.510994 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqmwk\" (UniqueName: \"kubernetes.io/projected/619a7b38-ba41-4dfd-b11a-bbcf1785d977-kube-api-access-tqmwk\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:30.864977 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:30.864936 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:38:31.016086 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.016009 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-offline-tls\") pod \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " Jun 08 12:38:31.016086 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.016046 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7klj\" (UniqueName: \"kubernetes.io/projected/fe2eb3a7-0253-41b0-99a1-81d4f8176548-kube-api-access-l7klj\") pod \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " Jun 08 12:38:31.016086 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.016080 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2eb3a7-0253-41b0-99a1-81d4f8176548-feast-data\") pod \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " Jun 08 12:38:31.016328 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.016113 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-online-tls\") pod \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " Jun 08 12:38:31.016328 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.016136 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-registry-tls\") pod \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " Jun 08 12:38:31.016328 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.016167 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-ui-tls\") pod \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\" (UID: \"fe2eb3a7-0253-41b0-99a1-81d4f8176548\") " Jun 08 12:38:31.016636 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.016604 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe2eb3a7-0253-41b0-99a1-81d4f8176548-feast-data" (OuterVolumeSpecName: "feast-data") pod "fe2eb3a7-0253-41b0-99a1-81d4f8176548" (UID: "fe2eb3a7-0253-41b0-99a1-81d4f8176548"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:38:31.018348 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.018320 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "fe2eb3a7-0253-41b0-99a1-81d4f8176548" (UID: "fe2eb3a7-0253-41b0-99a1-81d4f8176548"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:38:31.018455 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.018342 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-online-tls" (OuterVolumeSpecName: "online-tls") pod "fe2eb3a7-0253-41b0-99a1-81d4f8176548" (UID: "fe2eb3a7-0253-41b0-99a1-81d4f8176548"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:38:31.018455 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.018362 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fe2eb3a7-0253-41b0-99a1-81d4f8176548" (UID: "fe2eb3a7-0253-41b0-99a1-81d4f8176548"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:38:31.018649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.018629 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "fe2eb3a7-0253-41b0-99a1-81d4f8176548" (UID: "fe2eb3a7-0253-41b0-99a1-81d4f8176548"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:38:31.018649 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.018638 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe2eb3a7-0253-41b0-99a1-81d4f8176548-kube-api-access-l7klj" (OuterVolumeSpecName: "kube-api-access-l7klj") pod "fe2eb3a7-0253-41b0-99a1-81d4f8176548" (UID: "fe2eb3a7-0253-41b0-99a1-81d4f8176548"). InnerVolumeSpecName "kube-api-access-l7klj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:38:31.070316 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.070283 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerID="c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576" exitCode=137 Jun 08 12:38:31.070316 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.070307 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerID="1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52" exitCode=137 Jun 08 12:38:31.070467 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.070344 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" Jun 08 12:38:31.070467 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.070361 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerDied","Data":"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576"} Jun 08 12:38:31.070467 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.070402 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerDied","Data":"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52"} Jun 08 12:38:31.070467 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.070416 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg" event={"ID":"fe2eb3a7-0253-41b0-99a1-81d4f8176548","Type":"ContainerDied","Data":"8c1b8e9050aac8b08a215966e4e4ba48a0a3682a4a27c554ec91c896071ef666"} Jun 08 12:38:31.070467 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.070433 2580 scope.go:117] "RemoveContainer" containerID="74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204" Jun 08 12:38:31.072909 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.072884 2580 generic.go:356] "Generic (PLEG): container finished" podID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerID="7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b" exitCode=137 Jun 08 12:38:31.073037 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.072971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerDied","Data":"7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b"} Jun 08 12:38:31.073037 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.073011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" event={"ID":"619a7b38-ba41-4dfd-b11a-bbcf1785d977","Type":"ContainerDied","Data":"5f3e9b1f29a49a08da7ff2348fb273b05e7cdda1f4247266d0352e092d6a6bd1"} Jun 08 12:38:31.073037 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.072982 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj" Jun 08 12:38:31.079483 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.079461 2580 scope.go:117] "RemoveContainer" containerID="c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576" Jun 08 12:38:31.087060 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.087043 2580 scope.go:117] "RemoveContainer" containerID="73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379" Jun 08 12:38:31.094029 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.094007 2580 scope.go:117] "RemoveContainer" containerID="1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52" Jun 08 12:38:31.096822 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.096797 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg"] Jun 08 12:38:31.103482 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.103457 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-47gtg"] Jun 08 12:38:31.103723 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.103629 2580 scope.go:117] "RemoveContainer" containerID="e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e" Jun 08 12:38:31.116879 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.116855 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj"] Jun 08 12:38:31.117220 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.117192 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-offline-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:31.117293 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.117223 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7klj\" (UniqueName: \"kubernetes.io/projected/fe2eb3a7-0253-41b0-99a1-81d4f8176548-kube-api-access-l7klj\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:31.117293 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.117238 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2eb3a7-0253-41b0-99a1-81d4f8176548-feast-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:31.117293 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.117273 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-online-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:31.117293 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.117287 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:31.117477 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.117300 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe2eb3a7-0253-41b0-99a1-81d4f8176548-ui-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jun 08 12:38:31.122224 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.122201 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-b6qbj"] Jun 08 12:38:31.125067 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.125035 2580 scope.go:117] "RemoveContainer" containerID="68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61" Jun 08 12:38:31.131972 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.131955 2580 scope.go:117] "RemoveContainer" containerID="74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204" Jun 08 12:38:31.132222 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.132202 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204\": container with ID starting with 74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204 not found: ID does not exist" containerID="74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204" Jun 08 12:38:31.132267 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.132236 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204"} err="failed to get container status \"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204\": rpc error: code = NotFound desc = could not find container \"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204\": container with ID starting with 74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204 not found: ID does not exist" Jun 08 12:38:31.132267 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.132259 2580 scope.go:117] "RemoveContainer" containerID="c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576" Jun 08 12:38:31.132451 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.132436 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576\": container with ID starting with c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576 not found: ID does not exist" containerID="c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576" Jun 08 12:38:31.132499 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.132464 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576"} err="failed to get container status \"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576\": rpc error: code = NotFound desc = could not find container \"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576\": container with ID starting with c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576 not found: ID does not exist" Jun 08 12:38:31.132499 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.132478 2580 scope.go:117] "RemoveContainer" containerID="73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379" Jun 08 12:38:31.132712 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.132690 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379\": container with ID starting with 73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379 not found: ID does not exist" containerID="73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379" Jun 08 12:38:31.132779 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.132723 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379"} err="failed to get container status \"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379\": rpc error: code = NotFound desc = could not find container \"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379\": container with ID starting with 73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379 not found: ID does not exist" Jun 08 12:38:31.132779 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.132747 2580 scope.go:117] "RemoveContainer" containerID="1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52" Jun 08 12:38:31.132985 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.132966 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52\": container with ID starting with 1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52 not found: ID does not exist" containerID="1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52" Jun 08 12:38:31.133043 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.132989 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52"} err="failed to get container status \"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52\": rpc error: code = NotFound desc = could not find container \"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52\": container with ID starting with 1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52 not found: ID does not exist" Jun 08 12:38:31.133043 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133001 2580 scope.go:117] "RemoveContainer" containerID="e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e" Jun 08 12:38:31.133166 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.133152 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e\": container with ID starting with e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e not found: ID does not exist" containerID="e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e" Jun 08 12:38:31.133214 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133169 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e"} err="failed to get container status \"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e\": rpc error: code = NotFound desc = could not find container \"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e\": container with ID starting with e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e not found: ID does not exist" Jun 08 12:38:31.133214 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133179 2580 scope.go:117] "RemoveContainer" containerID="68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61" Jun 08 12:38:31.133370 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.133353 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61\": container with ID starting with 68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61 not found: ID does not exist" containerID="68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61" Jun 08 12:38:31.133415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133374 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61"} err="failed to get container status \"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61\": rpc error: code = NotFound desc = could not find container \"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61\": container with ID starting with 68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61 not found: ID does not exist" Jun 08 12:38:31.133415 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133388 2580 scope.go:117] "RemoveContainer" containerID="74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204" Jun 08 12:38:31.133556 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133536 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204"} err="failed to get container status \"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204\": rpc error: code = NotFound desc = could not find container \"74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204\": container with ID starting with 74db99634fcec6d500a71aa5ce741ad8f6cbd859934cb90369d1f8f7d3e13204 not found: ID does not exist" Jun 08 12:38:31.133626 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133558 2580 scope.go:117] "RemoveContainer" containerID="c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576" Jun 08 12:38:31.133780 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133764 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576"} err="failed to get container status \"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576\": rpc error: code = NotFound desc = could not find container \"c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576\": container with ID starting with c6d752a0816f122581d0a56e4c21163978295925d5b600944e55f56bbc553576 not found: ID does not exist" Jun 08 12:38:31.133824 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133780 2580 scope.go:117] "RemoveContainer" containerID="73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379" Jun 08 12:38:31.133989 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133968 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379"} err="failed to get container status \"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379\": rpc error: code = NotFound desc = could not find container \"73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379\": container with ID starting with 73cabd66132e27aed6c7100c86eff35c8f2aebd09ef91e97bdc3de2174db4379 not found: ID does not exist" Jun 08 12:38:31.134057 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.133990 2580 scope.go:117] "RemoveContainer" containerID="1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52" Jun 08 12:38:31.134210 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.134190 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52"} err="failed to get container status \"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52\": rpc error: code = NotFound desc = could not find container \"1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52\": container with ID starting with 1c5d4f3d30eca27f286dfc4926d40dc3e936e640278827f0a0d3f02f866cde52 not found: ID does not exist" Jun 08 12:38:31.134273 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.134212 2580 scope.go:117] "RemoveContainer" containerID="e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e" Jun 08 12:38:31.134411 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.134393 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e"} err="failed to get container status \"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e\": rpc error: code = NotFound desc = could not find container \"e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e\": container with ID starting with e1bc4a48234dbc79286994c33fc3c0b41fb4915c3c70c86509aaf63e9f6d0b8e not found: ID does not exist" Jun 08 12:38:31.134458 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.134412 2580 scope.go:117] "RemoveContainer" containerID="68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61" Jun 08 12:38:31.134563 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.134548 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61"} err="failed to get container status \"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61\": rpc error: code = NotFound desc = could not find container \"68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61\": container with ID starting with 68a60ced687f51ac2221314367c886f31fdad3943b55f8cb08efd35a72873f61 not found: ID does not exist" Jun 08 12:38:31.134613 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.134562 2580 scope.go:117] "RemoveContainer" containerID="3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794" Jun 08 12:38:31.141577 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.141562 2580 scope.go:117] "RemoveContainer" containerID="7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b" Jun 08 12:38:31.148217 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.148199 2580 scope.go:117] "RemoveContainer" containerID="aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6" Jun 08 12:38:31.155094 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.155078 2580 scope.go:117] "RemoveContainer" containerID="44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f" Jun 08 12:38:31.161388 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.161374 2580 scope.go:117] "RemoveContainer" containerID="ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8" Jun 08 12:38:31.167890 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.167873 2580 scope.go:117] "RemoveContainer" containerID="3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794" Jun 08 12:38:31.168136 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.168118 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794\": container with ID starting with 3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794 not found: ID does not exist" containerID="3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794" Jun 08 12:38:31.168202 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.168141 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794"} err="failed to get container status \"3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794\": rpc error: code = NotFound desc = could not find container \"3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794\": container with ID starting with 3cfea599b58a23b7e0e1073a89627b3efabe63d47ee6ec80e17cb950bd90b794 not found: ID does not exist" Jun 08 12:38:31.168202 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.168156 2580 scope.go:117] "RemoveContainer" containerID="7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b" Jun 08 12:38:31.168361 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.168345 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b\": container with ID starting with 7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b not found: ID does not exist" containerID="7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b" Jun 08 12:38:31.168413 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.168365 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b"} err="failed to get container status \"7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b\": rpc error: code = NotFound desc = could not find container \"7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b\": container with ID starting with 7a6e3fbba720e8cfd27ad4876f23d8330abecf21cb31a20465d396da7c0d562b not found: ID does not exist" Jun 08 12:38:31.168413 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.168378 2580 scope.go:117] "RemoveContainer" containerID="aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6" Jun 08 12:38:31.168614 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.168600 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6\": container with ID starting with aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6 not found: ID does not exist" containerID="aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6" Jun 08 12:38:31.168659 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.168616 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6"} err="failed to get container status \"aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6\": rpc error: code = NotFound desc = could not find container \"aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6\": container with ID starting with aed7de655655e8026e91f44f8981cec6afab06b7d9739c5e91bd7eca0d18d0e6 not found: ID does not exist" Jun 08 12:38:31.168659 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.168632 2580 scope.go:117] "RemoveContainer" containerID="44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f" Jun 08 12:38:31.168843 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.168825 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f\": container with ID starting with 44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f not found: ID does not exist" containerID="44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f" Jun 08 12:38:31.168889 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.168847 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f"} err="failed to get container status \"44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f\": rpc error: code = NotFound desc = could not find container \"44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f\": container with ID starting with 44b1112f4fbd753a48587d00710ea9c8a82e9266c38980dcf24ba3e37c0c509f not found: ID does not exist" Jun 08 12:38:31.168889 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.168861 2580 scope.go:117] "RemoveContainer" containerID="ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8" Jun 08 12:38:31.169104 ip-10-0-142-230 kubenswrapper[2580]: E0608 12:38:31.169086 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8\": container with ID starting with ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8 not found: ID does not exist" containerID="ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8" Jun 08 12:38:31.169161 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.169108 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8"} err="failed to get container status \"ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8\": rpc error: code = NotFound desc = could not find container \"ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8\": container with ID starting with ba50a8c84acabd6cb8ba445baa48296c08779f53fc39a28101402c6f26d5ecd8 not found: ID does not exist" Jun 08 12:38:31.859739 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.859710 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" path="/var/lib/kubelet/pods/619a7b38-ba41-4dfd-b11a-bbcf1785d977/volumes" Jun 08 12:38:31.860273 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:31.860257 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" path="/var/lib/kubelet/pods/fe2eb3a7-0253-41b0-99a1-81d4f8176548/volumes" Jun 08 12:38:43.494346 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:43.494270 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:44.772908 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:44.772879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:49.486891 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:49.486857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:49.593556 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:49.593513 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:50.189578 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:50.189548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:38:50.287597 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:38:50.287566 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:39:08.014773 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:08.014739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:39:08.175568 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:08.175531 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:39:08.278421 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:08.278345 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:39:10.942779 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:10.942747 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-446pr_80821cae-47d3-482a-8fdd-ac42bcf3e4f7/global-pull-secret-syncer/0.log" Jun 08 12:39:11.091577 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:11.091550 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" Jun 08 12:39:11.243078 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:11.243050 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-230.ec2.internal_ba29b9ee45638b00406cbe12817dc6fb/haproxy/0.log" Jun 08 12:39:13.579802 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:13.579766 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:39:13.660960 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:13.660914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bshpx_b5991d28-df9c-45fe-a16b-dc6a4d1a25bd/konnectivity-agent/0.log" limit=1024 Jun 08 12:39:15.403404 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.403371 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-dnrrb_2739c907-b011-4813-b88a-db69ed5688b1/monitoring-plugin/0.log" Jun 08 12:39:15.626000 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.625909 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xmw5n_0c1606b4-24e2-49a3-bc12-47e549a6c8d4/node-exporter/0.log" Jun 08 12:39:15.664225 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.664201 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xmw5n_0c1606b4-24e2-49a3-bc12-47e549a6c8d4/kube-rbac-proxy/0.log" Jun 08 12:39:15.709563 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.709544 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xmw5n_0c1606b4-24e2-49a3-bc12-47e549a6c8d4/init-textfile/0.log" Jun 08 12:39:15.741162 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.741135 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-lrl7l_1bc28b4b-9fe1-43f3-b9c8-2154200215bd/kube-rbac-proxy-main/0.log" Jun 08 12:39:15.766203 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.766179 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-lrl7l_1bc28b4b-9fe1-43f3-b9c8-2154200215bd/kube-rbac-proxy-self/0.log" Jun 08 12:39:15.793899 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.793882 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-lrl7l_1bc28b4b-9fe1-43f3-b9c8-2154200215bd/openshift-state-metrics/0.log" Jun 08 12:39:15.832290 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.832267 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce/prometheus/0.log" Jun 08 12:39:15.860117 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.860097 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce/config-reloader/0.log" Jun 08 12:39:15.886932 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.886886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce/thanos-sidecar/0.log" Jun 08 12:39:15.917741 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.917717 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce/kube-rbac-proxy-web/0.log" Jun 08 12:39:15.945566 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.945547 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce/kube-rbac-proxy/0.log" Jun 08 12:39:15.971894 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:15.971874 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce/kube-rbac-proxy-thanos/0.log" Jun 08 12:39:16.000337 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.000317 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e6c1cec-022f-4d5d-bbc1-7f1ddab87fce/init-config-reloader/0.log" Jun 08 12:39:16.036551 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.036523 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-ltnxx_8dfed933-9024-45d1-99e9-42bb7381cd43/prometheus-operator/0.log" Jun 08 12:39:16.072224 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.072204 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-ltnxx_8dfed933-9024-45d1-99e9-42bb7381cd43/kube-rbac-proxy/0.log" Jun 08 12:39:16.283078 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.283051 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7486f69c5-24lzt_3016476f-e848-49db-89be-0032c86f0c3a/thanos-query/0.log" Jun 08 12:39:16.320881 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.320859 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7486f69c5-24lzt_3016476f-e848-49db-89be-0032c86f0c3a/kube-rbac-proxy-web/0.log" Jun 08 12:39:16.355050 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.355030 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7486f69c5-24lzt_3016476f-e848-49db-89be-0032c86f0c3a/kube-rbac-proxy/0.log" Jun 08 12:39:16.394420 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.394395 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7486f69c5-24lzt_3016476f-e848-49db-89be-0032c86f0c3a/prom-label-proxy/0.log" Jun 08 12:39:16.429387 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.429363 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7486f69c5-24lzt_3016476f-e848-49db-89be-0032c86f0c3a/kube-rbac-proxy-rules/0.log" Jun 08 12:39:16.460320 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:16.460303 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7486f69c5-24lzt_3016476f-e848-49db-89be-0032c86f0c3a/kube-rbac-proxy-metrics/0.log" Jun 08 12:39:18.037609 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.037586 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/2.log" Jun 08 12:39:18.041933 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.041912 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-kvr8v_cacc1679-4496-434a-b777-29c9f516ac87/console-operator/3.log" Jun 08 12:39:18.520093 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520062 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg"] Jun 08 12:39:18.520346 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520335 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="registry" Jun 08 12:39:18.520391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520348 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="registry" Jun 08 12:39:18.520391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520357 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="online" Jun 08 12:39:18.520391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520362 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="online" Jun 08 12:39:18.520391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520371 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="ui" Jun 08 12:39:18.520391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520376 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="ui" Jun 08 12:39:18.520391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520383 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="feast-init" Jun 08 12:39:18.520391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520388 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="feast-init" Jun 08 12:39:18.520391 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520393 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="offline" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520399 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="offline" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520406 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="online" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520410 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="online" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520418 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="feast-apply" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520423 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="feast-apply" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520432 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="ui" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520437 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="ui" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520446 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="feast-init" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520451 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="feast-init" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520458 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="offline" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520463 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="offline" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520469 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="feast-apply" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520474 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="feast-apply" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520516 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="ui" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520526 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="online" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520533 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="offline" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520539 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="ui" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520544 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe2eb3a7-0253-41b0-99a1-81d4f8176548" containerName="registry" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520550 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="online" Jun 08 12:39:18.520635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.520557 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="619a7b38-ba41-4dfd-b11a-bbcf1785d977" containerName="offline" Jun 08 12:39:18.524706 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.524682 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.527521 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.527503 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdm9r\"/\"openshift-service-ca.crt\"" Jun 08 12:39:18.528497 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.528479 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jdm9r\"/\"default-dockercfg-vqvdl\"" Jun 08 12:39:18.528599 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.528509 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdm9r\"/\"kube-root-ca.crt\"" Jun 08 12:39:18.536090 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.536070 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg"] Jun 08 12:39:18.580241 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.580218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-sys\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.580344 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.580243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-lib-modules\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.580344 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.580279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-proc\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.580429 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.580375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-podres\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.580429 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.580401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29h6g\" (UniqueName: \"kubernetes.io/projected/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-kube-api-access-29h6g\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681422 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-sys\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-lib-modules\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-proc\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681561 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681512 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-sys\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681672 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681569 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-proc\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681672 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681573 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-podres\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681672 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29h6g\" (UniqueName: \"kubernetes.io/projected/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-kube-api-access-29h6g\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681672 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681625 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-lib-modules\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.681800 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.681704 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-podres\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.690331 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.690309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29h6g\" (UniqueName: \"kubernetes.io/projected/ffc6e5ed-7820-4dd1-bafd-438a7d08f349-kube-api-access-29h6g\") pod \"perf-node-gather-daemonset-dj4rg\" (UID: \"ffc6e5ed-7820-4dd1-bafd-438a7d08f349\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.834304 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.834225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:18.957353 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:18.957321 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg"] Jun 08 12:39:18.960386 ip-10-0-142-230 kubenswrapper[2580]: W0608 12:39:18.960358 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podffc6e5ed_7820_4dd1_bafd_438a7d08f349.slice/crio-2040fd6a066d29b3cec2eb057ec66a1b8f9870df036bb32c238c77e7382e28ba WatchSource:0}: Error finding container 2040fd6a066d29b3cec2eb057ec66a1b8f9870df036bb32c238c77e7382e28ba: Status 404 returned error can't find the container with id 2040fd6a066d29b3cec2eb057ec66a1b8f9870df036bb32c238c77e7382e28ba Jun 08 12:39:19.258228 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:19.258195 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" event={"ID":"ffc6e5ed-7820-4dd1-bafd-438a7d08f349","Type":"ContainerStarted","Data":"33f531e0efb44213c784721ead0a8afa3e050707a79bbcc578c75614ab0ccc60"} Jun 08 12:39:19.258635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:19.258246 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" event={"ID":"ffc6e5ed-7820-4dd1-bafd-438a7d08f349","Type":"ContainerStarted","Data":"2040fd6a066d29b3cec2eb057ec66a1b8f9870df036bb32c238c77e7382e28ba"} Jun 08 12:39:19.258635 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:19.258310 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:19.277160 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:19.277103 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" podStartSLOduration=1.277088414 podStartE2EDuration="1.277088414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:39:19.276085563 +0000 UTC m=+1921.915927009" watchObservedRunningTime="2026-06-08 12:39:19.277088414 +0000 UTC m=+1921.916929867" Jun 08 12:39:19.797492 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:19.797464 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8gqn6_55247bf6-9e28-41a7-b925-b8e43914ef45/dns/0.log" Jun 08 12:39:19.821274 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:19.821255 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8gqn6_55247bf6-9e28-41a7-b925-b8e43914ef45/kube-rbac-proxy/0.log" Jun 08 12:39:20.004690 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:20.004662 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w469r_e2b2638a-0ff1-4b0f-9d68-10d8278f33a2/dns-node-resolver/0.log" Jun 08 12:39:20.495828 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:20.495799 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fhmb9_66348e6c-fe0c-43c0-989c-fbd2c69373a1/node-ca/0.log" Jun 08 12:39:21.492932 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:21.492905 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5rmvq_636ec464-e603-4917-a35e-cdfa55a4491b/serve-healthcheck-canary/0.log" Jun 08 12:39:22.084828 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:22.084793 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wjt65_17f21fbe-654b-40d5-a3b4-5be67414fa01/kube-rbac-proxy/0.log" Jun 08 12:39:22.107150 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:22.107121 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wjt65_17f21fbe-654b-40d5-a3b4-5be67414fa01/exporter/0.log" Jun 08 12:39:22.130235 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:22.130209 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wjt65_17f21fbe-654b-40d5-a3b4-5be67414fa01/extractor/0.log" Jun 08 12:39:25.270300 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:25.270270 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-dj4rg" Jun 08 12:39:27.747854 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:27.747828 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9n5nt_48059286-7d75-419d-b43e-f7a83ee4fdb8/kube-multus-additional-cni-plugins/0.log" Jun 08 12:39:27.780143 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:27.780120 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9n5nt_48059286-7d75-419d-b43e-f7a83ee4fdb8/egress-router-binary-copy/0.log" Jun 08 12:39:27.813162 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:27.813144 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9n5nt_48059286-7d75-419d-b43e-f7a83ee4fdb8/cni-plugins/0.log" Jun 08 12:39:27.856209 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:27.856190 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9n5nt_48059286-7d75-419d-b43e-f7a83ee4fdb8/bond-cni-plugin/0.log" Jun 08 12:39:27.921671 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:27.921649 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9n5nt_48059286-7d75-419d-b43e-f7a83ee4fdb8/routeoverride-cni/0.log" Jun 08 12:39:27.991038 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:27.991016 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9n5nt_48059286-7d75-419d-b43e-f7a83ee4fdb8/whereabouts-cni-bincopy/0.log" Jun 08 12:39:28.043237 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:28.043214 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9n5nt_48059286-7d75-419d-b43e-f7a83ee4fdb8/whereabouts-cni/0.log" Jun 08 12:39:28.650911 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:28.650883 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d2jfx_419a276e-db0b-4bbb-acbb-31859a30e77c/kube-multus/0.log" Jun 08 12:39:28.831375 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:28.831348 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wr6xl_bfd9e9d8-9609-43d5-a45c-e58816b43118/network-metrics-daemon/0.log" Jun 08 12:39:28.867517 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:28.867497 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wr6xl_bfd9e9d8-9609-43d5-a45c-e58816b43118/kube-rbac-proxy/0.log" Jun 08 12:39:30.025905 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.025855 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-controller/0.log" Jun 08 12:39:30.048693 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.048613 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/0.log" Jun 08 12:39:30.050161 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.050145 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovn-acl-logging/1.log" Jun 08 12:39:30.079073 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.079051 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/kube-rbac-proxy-node/0.log" Jun 08 12:39:30.106778 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.106741 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/kube-rbac-proxy-ovn-metrics/0.log" Jun 08 12:39:30.128075 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.128041 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/northd/0.log" Jun 08 12:39:30.152084 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.152045 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/nbdb/0.log" Jun 08 12:39:30.175502 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.175452 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/sbdb/0.log" Jun 08 12:39:30.216151 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:30.216130 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j9m6c_0532dcdb-1611-488c-b8d3-6d84e84c687a/ovnkube-controller/0.log" Jun 08 12:39:31.413626 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:31.413600 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-9j7b8_6741b66e-5e5c-4be3-8f51-4b1093928dc0/network-check-target-container/0.log" Jun 08 12:39:32.487238 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:32.487211 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-js9c4_f9581e35-486d-4e40-b4e7-96507433032c/iptables-alerter/0.log" Jun 08 12:39:33.210196 ip-10-0-142-230 kubenswrapper[2580]: I0608 12:39:33.210161 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-wmwnv_a70de4b5-5a90-45de-a1a4-9a78ef773ba1/tuned/0.log"