Jun 10 12:45:31.153146 ip-10-0-140-124 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 10 12:45:31.153160 ip-10-0-140-124 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 10 12:45:31.153170 ip-10-0-140-124 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 10 12:45:31.153509 ip-10-0-140-124 systemd[1]: Failed to start Kubernetes Kubelet. Jun 10 12:45:41.638784 ip-10-0-140-124 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jun 10 12:45:41.638884 ip-10-0-140-124 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a655201d242b46baad817dce50f0e72c -- Jun 10 12:48:04.166100 ip-10-0-140-124 systemd[1]: Starting Kubernetes Kubelet... Jun 10 12:48:04.615183 ip-10-0-140-124 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:48:04.615183 ip-10-0-140-124 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 10 12:48:04.615183 ip-10-0-140-124 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:48:04.615183 ip-10-0-140-124 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 10 12:48:04.615183 ip-10-0-140-124 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:48:04.616184 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.616096 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 10 12:48:04.622521 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622501 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:04.622521 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622517 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:04.622521 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622523 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:04.622521 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622527 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:04.622521 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622531 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622536 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622543 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622550 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622556 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622559 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622563 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622567 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622570 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622574 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622578 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622582 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622586 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622590 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622594 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622598 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622603 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622607 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622612 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:04.622842 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622616 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622621 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622625 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622629 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622633 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622637 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622640 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622644 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622648 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622652 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622656 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622661 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622665 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622669 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622674 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622679 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622684 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622688 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622692 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622697 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:04.623607 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622701 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622707 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622711 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622715 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622720 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622725 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622729 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622733 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622737 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622741 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622749 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622755 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622760 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622764 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622768 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622773 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622777 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622781 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622785 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622789 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:04.624470 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622794 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622798 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622817 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622822 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622826 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622831 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622835 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622841 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622846 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622851 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622855 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622860 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622864 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622869 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622874 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622879 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622883 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622887 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622891 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:04.625366 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622896 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622900 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622907 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622911 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622915 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622920 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622924 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622931 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622937 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622942 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622947 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622951 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622956 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622962 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622966 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622970 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.622974 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624172 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624182 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624187 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:04.626195 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624195 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624201 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624206 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624211 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624215 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624220 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624224 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624228 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624237 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624242 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624246 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624251 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624255 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624262 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624268 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624272 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624277 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624282 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624286 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:04.627056 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624291 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624295 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624300 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624304 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624308 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624313 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624317 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624321 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624326 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624330 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624335 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624339 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624343 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624348 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624354 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624358 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624362 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624369 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624374 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:04.627673 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624379 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624383 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624388 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624393 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624398 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624402 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624406 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624412 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624417 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624421 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624425 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624429 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624433 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624438 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624442 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624446 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624450 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624455 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624459 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:04.628169 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624463 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624466 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624471 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624475 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624479 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624482 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624486 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624491 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624496 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624500 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624505 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624509 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624513 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624518 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624522 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624526 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624531 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624536 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624540 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624544 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:04.628730 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624548 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624553 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624557 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624561 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624565 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624569 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624574 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624578 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624582 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624586 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624596 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624600 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624604 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624608 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624613 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624618 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624622 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624625 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.624630 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625268 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:04.629365 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625277 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625282 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625286 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625291 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625298 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625304 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625309 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625314 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625319 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625324 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625328 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625333 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625337 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625342 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625348 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625354 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625359 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625363 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625367 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:04.629998 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625370 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625374 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625378 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625384 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625388 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625392 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625396 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625401 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625405 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625409 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625413 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625418 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625423 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625427 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625432 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625437 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625441 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625446 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625450 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:04.630475 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625454 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625458 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625463 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625467 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625473 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625477 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625481 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625485 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625490 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625494 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625498 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625502 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625507 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625512 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625517 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625521 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625527 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625531 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625536 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625540 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:04.631066 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625545 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625549 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625553 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625557 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625562 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625567 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625571 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625575 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625579 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625585 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625590 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625594 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625597 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625602 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625606 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625610 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625615 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625619 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625624 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625627 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:04.631596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625634 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625638 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625642 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625646 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625650 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625654 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625659 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625663 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625669 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625673 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625677 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625681 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625685 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625689 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625694 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625698 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625703 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625707 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625711 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.625716 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625841 2577 flags.go:64] FLAG: --address="0.0.0.0" Jun 10 12:48:04.632238 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625859 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625870 2577 flags.go:64] FLAG: --anonymous-auth="true" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625877 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625883 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625889 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625895 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625902 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625907 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625912 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625918 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625924 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625931 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625936 2577 flags.go:64] FLAG: --cgroup-root="" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625941 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625946 2577 flags.go:64] FLAG: --client-ca-file="" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625950 2577 flags.go:64] FLAG: --cloud-provider="external" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625955 2577 flags.go:64] FLAG: --cluster-dns="[]" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625962 2577 flags.go:64] FLAG: --cluster-domain="" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625967 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625973 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625978 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625984 2577 flags.go:64] FLAG: --container-log-max-files="5" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625990 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 10 12:48:04.633076 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.625994 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626000 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626005 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626010 2577 flags.go:64] FLAG: --contention-profiling="false" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626015 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626020 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626025 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626029 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626036 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626041 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626046 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626051 2577 flags.go:64] FLAG: --enable-load-reader="false" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626055 2577 flags.go:64] FLAG: --enable-server="true" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626060 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626068 2577 flags.go:64] FLAG: --event-burst="100" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626072 2577 flags.go:64] FLAG: --event-qps="50" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626077 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626082 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626087 2577 flags.go:64] FLAG: --eviction-hard="" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626094 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626101 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626106 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626111 2577 flags.go:64] FLAG: --eviction-soft="" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626115 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626121 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 10 12:48:04.633999 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626126 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626131 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626135 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626141 2577 flags.go:64] FLAG: --fail-swap-on="true" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626146 2577 flags.go:64] FLAG: --feature-gates="" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626152 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626157 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626162 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626168 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626173 2577 flags.go:64] FLAG: --healthz-port="10248" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626178 2577 flags.go:64] FLAG: --help="false" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626183 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626188 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626192 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626197 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626202 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626208 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626213 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626217 2577 flags.go:64] FLAG: --image-service-endpoint="" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626222 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626226 2577 flags.go:64] FLAG: --kube-api-burst="100" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626231 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626237 2577 flags.go:64] FLAG: --kube-api-qps="50" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626241 2577 flags.go:64] FLAG: --kube-reserved="" Jun 10 12:48:04.635075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626246 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626251 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626257 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626264 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626269 2577 flags.go:64] FLAG: --lock-file="" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626273 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626278 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626284 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626292 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626297 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626302 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626307 2577 flags.go:64] FLAG: --logging-format="text" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626312 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626317 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626322 2577 flags.go:64] FLAG: --manifest-url="" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626327 2577 flags.go:64] FLAG: --manifest-url-header="" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626334 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626339 2577 flags.go:64] FLAG: --max-open-files="1000000" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626345 2577 flags.go:64] FLAG: --max-pods="110" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626350 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626354 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626359 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626371 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626377 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626381 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 10 12:48:04.635747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626387 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626398 2577 flags.go:64] FLAG: --node-status-max-images="50" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626403 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626408 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626413 2577 flags.go:64] FLAG: --pod-cidr="" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626418 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626427 2577 flags.go:64] FLAG: --pod-manifest-path="" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626431 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626436 2577 flags.go:64] FLAG: --pods-per-core="0" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626441 2577 flags.go:64] FLAG: --port="10250" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626448 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626454 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-044f761b7aeb6a5a5" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626459 2577 flags.go:64] FLAG: --qos-reserved="" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626464 2577 flags.go:64] FLAG: --read-only-port="10255" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626469 2577 flags.go:64] FLAG: --register-node="true" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626474 2577 flags.go:64] FLAG: --register-with-taints="" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626480 2577 flags.go:64] FLAG: --registry-burst="10" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626484 2577 flags.go:64] FLAG: --registry-qps="5" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626489 2577 flags.go:64] FLAG: --reserved-cpus="" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626494 2577 flags.go:64] FLAG: --reserved-memory="" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626500 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626505 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626510 2577 flags.go:64] FLAG: --rotate-certificates="false" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626515 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626519 2577 flags.go:64] FLAG: --runonce="false" Jun 10 12:48:04.636450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626524 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626529 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626534 2577 flags.go:64] FLAG: --seccomp-default="false" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626539 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626544 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626549 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626554 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626559 2577 flags.go:64] FLAG: --storage-driver-password="root" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626564 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626569 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626574 2577 flags.go:64] FLAG: --storage-driver-user="root" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626578 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626583 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626588 2577 flags.go:64] FLAG: --system-cgroups="" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626593 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626601 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626606 2577 flags.go:64] FLAG: --tls-cert-file="" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626613 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626620 2577 flags.go:64] FLAG: --tls-min-version="" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626635 2577 flags.go:64] FLAG: --tls-private-key-file="" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626640 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626645 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626650 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626655 2577 flags.go:64] FLAG: --v="2" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626661 2577 flags.go:64] FLAG: --version="false" Jun 10 12:48:04.637157 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626668 2577 flags.go:64] FLAG: --vmodule="" Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626674 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.626679 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626882 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626890 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626895 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626900 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626904 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626909 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626915 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626921 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626926 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626931 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626936 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626941 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626945 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626950 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626954 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626958 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:04.637853 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626963 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626967 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626972 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626976 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626980 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626986 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626991 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.626995 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627000 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627005 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627009 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627013 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627017 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627022 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627027 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627031 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627036 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627040 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627044 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627048 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:04.638380 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627052 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627056 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627060 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627064 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627068 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627072 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627076 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627080 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627085 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627089 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627093 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627097 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627102 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627106 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627113 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627118 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627123 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627130 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627134 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:04.638937 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627139 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627144 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627149 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627154 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627159 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627163 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627167 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627174 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627179 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627183 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627188 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627192 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627196 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627200 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627205 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627209 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627213 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627217 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627222 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627226 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:04.639402 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627231 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627236 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627240 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627245 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627249 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627253 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627258 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627262 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627266 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627270 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627277 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627281 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627286 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627290 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627294 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627303 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627308 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627312 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627316 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627322 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:04.639945 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627326 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627332 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627339 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.627344 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.628248 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.636452 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.636468 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.636489 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.636494 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636555 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636560 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636563 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636566 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636570 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636574 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:04.640444 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636577 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636580 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636583 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636585 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636588 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636591 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636593 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636596 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636599 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636601 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636604 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636607 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636609 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636612 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636614 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636617 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636620 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636622 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636625 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:04.640965 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636629 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636633 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636636 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636639 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636644 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636647 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636650 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636653 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636656 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636658 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636662 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636664 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636667 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636669 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636672 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636675 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636679 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636682 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636685 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:04.641426 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636688 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636690 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636693 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636696 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636698 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636701 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636703 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636706 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636709 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636712 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636714 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636716 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636719 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636721 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636724 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636726 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636729 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636734 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636738 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636742 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:04.641918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636744 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636747 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636750 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636752 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636755 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636757 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636760 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636762 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636765 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636767 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636770 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636772 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636775 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636777 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636779 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636782 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636785 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636787 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636790 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636793 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:04.642419 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636795 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636798 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636816 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636819 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636822 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636825 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636827 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636830 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636833 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636836 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636839 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636842 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636845 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636848 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636850 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:04.642931 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.636855 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636973 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636979 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636982 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636985 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636988 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636991 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636993 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636996 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.636999 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637001 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637004 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637006 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637009 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637012 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637014 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637017 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637019 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637022 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637024 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:04.643298 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637027 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637029 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637032 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637035 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637037 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637040 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637042 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637046 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637049 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637052 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637054 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637057 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637059 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637062 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637065 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637067 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637070 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637072 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637075 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637077 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:04.643773 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637080 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637083 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637085 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637088 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637090 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637093 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637095 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637098 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637101 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637103 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637106 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637108 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637111 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637113 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637116 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637119 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637122 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637125 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637128 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637130 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:04.644282 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637133 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637136 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637139 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637141 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637144 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637146 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637150 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637154 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637157 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637160 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637183 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637187 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637190 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637193 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637196 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637199 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637201 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637204 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637207 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:04.644988 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637210 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637212 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637215 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637219 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637223 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637226 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637230 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637233 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637236 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637239 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637242 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637245 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637247 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637250 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637253 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637255 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637258 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637261 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637264 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:04.645700 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637266 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:04.646478 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:04.637269 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:04.646478 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.637273 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:48:04.646478 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.637411 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jun 10 12:48:04.646478 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.639850 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 10 12:48:04.646478 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.641557 2577 server.go:1019] "Starting client certificate rotation" Jun 10 12:48:04.646478 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.641660 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 12:48:04.646478 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.641695 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 12:48:04.646838 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.646820 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 12:48:04.647989 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.647968 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 12:48:04.665493 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.665462 2577 log.go:25] "Validated CRI v1 runtime API" Jun 10 12:48:04.668551 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.668534 2577 log.go:25] "Validated CRI v1 image API" Jun 10 12:48:04.670320 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.670300 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 10 12:48:04.672281 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.672264 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 12:48:04.673781 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.673755 2577 fs.go:135] Filesystem UUIDs: map[15f4390c-d317-4b8b-b6d2-9c920de6eee6:/dev/nvme0n1p3 55adb989-25cf-414a-b143-b5e2a7cca9e3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 10 12:48:04.673858 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.673780 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 10 12:48:04.680646 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.680535 2577 manager.go:216] Machine: {Timestamp:2026-06-10 12:48:04.679168823 +0000 UTC m=+0.385473005 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093613 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2831cc652c258312a3141fcd1f2c46 SystemUUID:ec2831cc-652c-2583-12a3-141fcd1f2c46 BootID:a655201d-242b-46ba-ad81-7dce50f0e72c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fb:35:53:10:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fb:35:53:10:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:6c:8c:c7:02:b7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 10 12:48:04.680646 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.680640 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 10 12:48:04.680762 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.680722 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 10 12:48:04.684285 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.684257 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 10 12:48:04.684431 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.684288 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-124.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 10 12:48:04.685136 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.685127 2577 topology_manager.go:138] "Creating topology manager with none policy" Jun 10 12:48:04.685168 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.685138 2577 container_manager_linux.go:309] "Creating device plugin manager" Jun 10 12:48:04.685168 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.685146 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 12:48:04.685963 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.685953 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 12:48:04.687226 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.687216 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 10 12:48:04.691706 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.691695 2577 state_mem.go:36] "Initialized new in-memory state store" Jun 10 12:48:04.691846 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.691836 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 10 12:48:04.691876 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.691868 2577 kubelet.go:486] "Attempting to sync node with API server" Jun 10 12:48:04.691903 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.691878 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 10 12:48:04.691903 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.691892 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 10 12:48:04.691903 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.691903 2577 kubelet.go:398] "Adding apiserver pod source" Jun 10 12:48:04.691976 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.691912 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 10 12:48:04.692375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.692352 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 12:48:04.692474 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.692382 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 12:48:04.695323 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.695305 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 10 12:48:04.697658 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.697638 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 10 12:48:04.697743 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.697668 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699686 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699723 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699736 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699746 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699760 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699770 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699790 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699814 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699827 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 10 12:48:04.699828 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699836 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 10 12:48:04.700251 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699867 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 10 12:48:04.700251 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.699904 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 10 12:48:04.700937 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.700915 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 10 12:48:04.701012 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.700965 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 10 12:48:04.702496 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.702476 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rmv59" Jun 10 12:48:04.704404 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.704382 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-124.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 10 12:48:04.704786 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.704772 2577 server.go:1268] "Started kubelet" Jun 10 12:48:04.704897 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.704867 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 10 12:48:04.704995 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.704958 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 10 12:48:04.704995 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.705016 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 10 12:48:04.705153 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.705037 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 10 12:48:04.705153 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.705053 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-124.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 10 12:48:04.705581 ip-10-0-140-124 systemd[1]: Started Kubernetes Kubelet. Jun 10 12:48:04.706934 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.706921 2577 server.go:310] "Adding debug handlers to kubelet server" Jun 10 12:48:04.707341 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.707317 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 10 12:48:04.710625 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.710606 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rmv59" Jun 10 12:48:04.711063 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.710017 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-124.ec2.internal.18b7b916adcea61b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-124.ec2.internal,UID:ip-10-0-140-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-124.ec2.internal,},FirstTimestamp:2026-06-10 12:48:04.704740891 +0000 UTC m=+0.411045071,LastTimestamp:2026-06-10 12:48:04.704740891 +0000 UTC m=+0.411045071,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-124.ec2.internal,}" Jun 10 12:48:04.712432 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.712415 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 10 12:48:04.714839 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.714821 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 10 12:48:04.714839 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.714797 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 10 12:48:04.714980 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.714948 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:04.715534 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.715513 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 10 12:48:04.715534 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.715514 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 10 12:48:04.715534 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.715540 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 10 12:48:04.715754 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.715653 2577 reconstruct.go:97] "Volume reconstruction finished" Jun 10 12:48:04.715754 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.715664 2577 reconciler.go:29] "Reconciler: start to sync state" Jun 10 12:48:04.715963 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.715947 2577 factory.go:153] Registering CRI-O factory Jun 10 12:48:04.715963 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.715963 2577 factory.go:223] Registration of the crio container factory successfully Jun 10 12:48:04.716112 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.716014 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 10 12:48:04.716112 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.716021 2577 factory.go:55] Registering systemd factory Jun 10 12:48:04.716112 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.716027 2577 factory.go:223] Registration of the systemd container factory successfully Jun 10 12:48:04.716112 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.716046 2577 factory.go:103] Registering Raw factory Jun 10 12:48:04.716112 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.716080 2577 manager.go:1196] Started watching for new ooms in manager Jun 10 12:48:04.716446 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.716435 2577 manager.go:349] Starting recovery of all containers Jun 10 12:48:04.725407 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.725245 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:04.726357 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.726335 2577 manager.go:354] Recovery completed Jun 10 12:48:04.728398 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.728372 2577 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 10 12:48:04.729881 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.729862 2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-124.ec2.internal\" not found" node="ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.732130 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.732117 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:04.735495 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.735481 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:04.735557 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.735508 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:04.735557 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.735518 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:04.736064 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.736051 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 10 12:48:04.736064 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.736062 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 10 12:48:04.736163 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.736064 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:04.736163 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.736079 2577 state_mem.go:36] "Initialized new in-memory state store" Jun 10 12:48:04.737006 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.736992 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:04.737074 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.737018 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:04.737074 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.737028 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:04.738071 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.738061 2577 policy_none.go:49] "None policy: Start" Jun 10 12:48:04.738111 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.738076 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 10 12:48:04.738111 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.738086 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 10 12:48:04.739818 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.739797 2577 policy_none.go:47] "Start" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.777407 2577 manager.go:337] "Starting Device Plugin manager" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.777444 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.777457 2577 server.go:87] "Starting device plugin registration server" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.777733 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.777749 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.777859 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.777929 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.777938 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.778524 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.778577 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.784296 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.785465 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.785495 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.785513 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.785548 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 10 12:48:04.795232 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.789021 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:04.878645 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.878558 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:04.879593 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.879574 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:04.879663 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.879611 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:04.879663 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.879622 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:04.879663 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.879647 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.886480 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.886463 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal"] Jun 10 12:48:04.886529 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.886523 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:04.888022 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.888003 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:04.888075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.888035 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:04.888075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.888045 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:04.888321 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.888308 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.888354 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.888328 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-124.ec2.internal\": node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:04.890130 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.890115 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:04.890265 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.890242 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.890329 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.890276 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:04.891150 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.891132 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:04.891239 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.891164 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:04.891239 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.891178 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:04.891239 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.891132 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:04.891377 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.891248 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:04.891377 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.891261 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:04.893214 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.893196 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.893304 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.893226 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:04.894046 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.894030 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:04.894104 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.894064 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:04.894104 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.894076 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:04.904881 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.904861 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:04.916943 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.916917 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a301da613e9794c1472f3518634c2dc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal\" (UID: \"a301da613e9794c1472f3518634c2dc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.917034 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.916952 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a301da613e9794c1472f3518634c2dc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal\" (UID: \"a301da613e9794c1472f3518634c2dc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.917034 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:04.916977 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/465107ac2ceb7852c065a227546cfcdf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-124.ec2.internal\" (UID: \"465107ac2ceb7852c065a227546cfcdf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.925549 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.925524 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-124.ec2.internal\" not found" node="ip-10-0-140-124.ec2.internal" Jun 10 12:48:04.928831 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:04.928799 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-124.ec2.internal\" not found" node="ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.005886 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.005858 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.017143 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.017118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a301da613e9794c1472f3518634c2dc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal\" (UID: \"a301da613e9794c1472f3518634c2dc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.017238 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.017148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a301da613e9794c1472f3518634c2dc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal\" (UID: \"a301da613e9794c1472f3518634c2dc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.017238 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.017165 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/465107ac2ceb7852c065a227546cfcdf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-124.ec2.internal\" (UID: \"465107ac2ceb7852c065a227546cfcdf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.017238 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.017210 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/465107ac2ceb7852c065a227546cfcdf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-124.ec2.internal\" (UID: \"465107ac2ceb7852c065a227546cfcdf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.017238 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.017214 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a301da613e9794c1472f3518634c2dc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal\" (UID: \"a301da613e9794c1472f3518634c2dc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.017387 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.017243 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a301da613e9794c1472f3518634c2dc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal\" (UID: \"a301da613e9794c1472f3518634c2dc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.106959 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.106928 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.207764 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.207663 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.231111 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.231088 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.234667 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.234644 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" Jun 10 12:48:05.308020 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.307980 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.408473 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.408449 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.509025 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.508947 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.609382 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.609355 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.640914 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.640884 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 10 12:48:05.641517 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.641026 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 10 12:48:05.641517 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.641060 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 10 12:48:05.710268 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.710235 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.717817 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.717770 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-09 12:43:04 +0000 UTC" deadline="2027-12-11 17:00:24.761132105 +0000 UTC" Jun 10 12:48:05.717817 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.717796 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13180h12m19.04333936s" Jun 10 12:48:05.794875 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:05.794837 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465107ac2ceb7852c065a227546cfcdf.slice/crio-4100a2b23c62e7ff0474c77c4c1fabe8edfad70cb34399f4f467017b1911e7e6 WatchSource:0}: Error finding container 4100a2b23c62e7ff0474c77c4c1fabe8edfad70cb34399f4f467017b1911e7e6: Status 404 returned error can't find the container with id 4100a2b23c62e7ff0474c77c4c1fabe8edfad70cb34399f4f467017b1911e7e6 Jun 10 12:48:05.795523 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:05.795497 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda301da613e9794c1472f3518634c2dc6.slice/crio-a4e01ea442cced7946ac97370389229b88f92e8985121db4db830654f78fc015 WatchSource:0}: Error finding container a4e01ea442cced7946ac97370389229b88f92e8985121db4db830654f78fc015: Status 404 returned error can't find the container with id a4e01ea442cced7946ac97370389229b88f92e8985121db4db830654f78fc015 Jun 10 12:48:05.799404 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.799379 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:48:05.810989 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.810965 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.911905 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:05.911863 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:05.955992 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:05.955959 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:06.012327 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.012297 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:06.112919 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.112839 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-124.ec2.internal\" not found" Jun 10 12:48:06.180095 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.180063 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:06.216068 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.215868 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" Jun 10 12:48:06.240974 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.240945 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 12:48:06.242491 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.242221 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" Jun 10 12:48:06.261074 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.261048 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 12:48:06.498986 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.498897 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:06.545736 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.545706 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:06.692497 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.692464 2577 apiserver.go:52] "Watching apiserver" Jun 10 12:48:06.702851 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.702825 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 10 12:48:06.703557 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.703530 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-tvzvq","openshift-ovn-kubernetes/ovnkube-node-zqwjk","kube-system/global-pull-secret-syncer-k5ktc","kube-system/konnectivity-agent-j8bc8","kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9","openshift-cluster-node-tuning-operator/tuned-tv4cv","openshift-image-registry/node-ca-k7tz2","openshift-multus/multus-z22dk","openshift-multus/network-metrics-daemon-gcbmf","openshift-dns/node-resolver-wqrsz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal","openshift-multus/multus-additional-cni-plugins-bz7mt","openshift-network-diagnostics/network-check-target-v8wvp"] Jun 10 12:48:06.706226 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.706204 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:06.706326 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.706288 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:06.708243 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.708220 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.710378 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.710353 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:06.710475 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.710424 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:06.710949 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.710914 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 10 12:48:06.711061 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.710920 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ksn79\"" Jun 10 12:48:06.711176 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.711088 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 10 12:48:06.712494 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.712475 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.714603 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.714585 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.715011 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.714986 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 10 12:48:06.715011 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.715006 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 10 12:48:06.715177 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.715034 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 10 12:48:06.715177 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.715052 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qsrp6\"" Jun 10 12:48:06.715177 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.715128 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 10 12:48:06.716877 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.716847 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.717395 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.717196 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 10 12:48:06.717395 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.717338 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:48:06.717395 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.717352 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8b5cb\"" Jun 10 12:48:06.718989 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.718971 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.719648 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.719627 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 10 12:48:06.719745 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.719681 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 10 12:48:06.720253 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.720234 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wvbf9\"" Jun 10 12:48:06.720343 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.720236 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 10 12:48:06.721177 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.721163 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.723205 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.723103 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 10 12:48:06.723205 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.723167 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 10 12:48:06.723461 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.723366 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 10 12:48:06.723461 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.723432 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lth5x\"" Jun 10 12:48:06.723596 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.723469 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 10 12:48:06.723646 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.723628 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.726125 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.726110 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.726670 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.726650 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 10 12:48:06.727169 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727150 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tm58f\"" Jun 10 12:48:06.727267 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727227 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sjq7\" (UniqueName: \"kubernetes.io/projected/1d88de4b-5158-4a97-b796-74a12f088e70-kube-api-access-8sjq7\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:06.727328 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cd4f44a-0a1e-4710-a68c-be170c527c22-hosts-file\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.727405 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727385 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-run\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.727507 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727397 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 10 12:48:06.727507 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727496 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-var-lib-kubelet\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.727622 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727552 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8136e2a8-4ddc-46fa-b368-c6803750c135-host\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.727622 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727398 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q4n4d\"" Jun 10 12:48:06.727622 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727576 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-systemd\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.727793 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727636 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th77c\" (UniqueName: \"kubernetes.io/projected/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-kube-api-access-th77c\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.727793 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727664 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-device-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.727793 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727687 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-modprobe-d\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.727793 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727704 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 10 12:48:06.727793 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysctl-d\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.727793 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysctl-conf\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.728126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-registration-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.728126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gwxw\" (UniqueName: \"kubernetes.io/projected/559de811-5e29-47d5-8dd3-91e74a082d76-kube-api-access-7gwxw\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.728126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727872 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 10 12:48:06.728126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727891 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:06.728126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727952 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-socket-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.728126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.727955 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 10 12:48:06.728126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysconfig\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.728126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728101 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-host\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728129 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-tuned\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-sys-fs\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728164 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cd4f44a-0a1e-4710-a68c-be170c527c22-tmp-dir\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728183 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkcbg\" (UniqueName: \"kubernetes.io/projected/1cd4f44a-0a1e-4710-a68c-be170c527c22-kube-api-access-jkcbg\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728205 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-etc-selinux\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728227 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqlgg\" (UniqueName: \"kubernetes.io/projected/8136e2a8-4ddc-46fa-b368-c6803750c135-kube-api-access-hqlgg\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-kubernetes\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-sys\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728329 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8136e2a8-4ddc-46fa-b368-c6803750c135-serviceca\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728393 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-lib-modules\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728426 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-tmp\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.728515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728461 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.729121 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.728772 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 10 12:48:06.729121 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.729012 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 10 12:48:06.729121 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.729034 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:06.729281 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.729258 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 10 12:48:06.729705 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.729685 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:48:06.729842 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.729746 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k2jsf\"" Jun 10 12:48:06.730449 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.730423 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 10 12:48:06.731681 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.731641 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 10 12:48:06.732142 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.732123 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 10 12:48:06.733304 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.733271 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.733450 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.733427 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:06.734395 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.734376 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 10 12:48:06.734738 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.734721 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 12:48:06.738608 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.738105 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-psfkb\"" Jun 10 12:48:06.765092 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.765069 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xpzfb" Jun 10 12:48:06.771155 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.771135 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xpzfb" Jun 10 12:48:06.790412 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.790356 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" event={"ID":"a301da613e9794c1472f3518634c2dc6","Type":"ContainerStarted","Data":"a4e01ea442cced7946ac97370389229b88f92e8985121db4db830654f78fc015"} Jun 10 12:48:06.791420 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.791389 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" event={"ID":"465107ac2ceb7852c065a227546cfcdf","Type":"ContainerStarted","Data":"4100a2b23c62e7ff0474c77c4c1fabe8edfad70cb34399f4f467017b1911e7e6"} Jun 10 12:48:06.816428 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.816402 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 10 12:48:06.829598 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-socket-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.829598 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-hostroot\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.829780 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829616 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-kubelet\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.829780 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829632 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab-konnectivity-ca\") pod \"konnectivity-agent-j8bc8\" (UID: \"6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab\") " pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:06.829780 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-host\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.829780 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-tuned\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.829780 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829767 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-sys-fs\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.830031 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829799 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-system-cni-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.830031 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829832 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-host\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.830031 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829865 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-sys-fs\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.830031 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829864 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16a5c16f-7a51-4717-b623-fb1f604add4c-cni-binary-copy\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.830031 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829920 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-k8s-cni-cncf-io\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.830031 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829950 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-socket-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.830031 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.829958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcj7g\" (UniqueName: \"kubernetes.io/projected/16a5c16f-7a51-4717-b623-fb1f604add4c-kube-api-access-vcj7g\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.830031 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830042 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-slash\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-etc-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830116 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-cni-bin\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-kubernetes\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830191 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8136e2a8-4ddc-46fa-b368-c6803750c135-serviceca\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830217 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-cnibin\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830252 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-kubernetes\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-var-lib-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830290 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sjq7\" (UniqueName: \"kubernetes.io/projected/1d88de4b-5158-4a97-b796-74a12f088e70-kube-api-access-8sjq7\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830310 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cd4f44a-0a1e-4710-a68c-be170c527c22-hosts-file\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830344 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-run\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.830375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-cni-bin\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830393 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovnkube-config\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab-agent-certs\") pod \"konnectivity-agent-j8bc8\" (UID: \"6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab\") " pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830444 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cd4f44a-0a1e-4710-a68c-be170c527c22-hosts-file\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-systemd\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-run-netns\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830537 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830561 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-run\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830596 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-systemd\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830650 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysctl-d\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830680 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysctl-conf\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8136e2a8-4ddc-46fa-b368-c6803750c135-serviceca\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830706 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-cni-netd\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830730 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7cac882-e105-4225-bc58-ee9f8b8f22f4-host-slash\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830784 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-kubelet\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.831269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830825 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-multus-certs\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830848 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysctl-conf\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.830909 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.830920 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovn-node-metrics-cert\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.830963 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:07.330948039 +0000 UTC m=+3.037252209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysconfig\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831034 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysctl-d\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-os-release\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831083 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-netns\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831141 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-sysconfig\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831187 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-systemd-units\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831213 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbeeb78c-7022-4a4e-8834-767fef6bb920-dbus\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cd4f44a-0a1e-4710-a68c-be170c527c22-tmp-dir\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831267 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkcbg\" (UniqueName: \"kubernetes.io/projected/1cd4f44a-0a1e-4710-a68c-be170c527c22-kube-api-access-jkcbg\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831292 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-etc-selinux\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-cni-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.831911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831339 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-cni-multus\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-etc-kubernetes\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-os-release\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831390 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-etc-selinux\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831410 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-systemd\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831435 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqlgg\" (UniqueName: \"kubernetes.io/projected/8136e2a8-4ddc-46fa-b368-c6803750c135-kube-api-access-hqlgg\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-sys\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831485 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-conf-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831496 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cd4f44a-0a1e-4710-a68c-be170c527c22-tmp-dir\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831511 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-daemon-config\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831550 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hqhf\" (UniqueName: \"kubernetes.io/projected/213701d6-76f7-4d6e-8012-6918d482c2c9-kube-api-access-5hqhf\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831575 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-node-log\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831590 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-sys\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831599 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t79j5\" (UniqueName: \"kubernetes.io/projected/18ff83ff-de0e-418c-bab6-cdbf795df6ce-kube-api-access-t79j5\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831623 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7cac882-e105-4225-bc58-ee9f8b8f22f4-iptables-alerter-script\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831647 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831698 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:06.832720 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-lib-modules\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831751 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-tmp\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831775 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-socket-dir-parent\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-ovn\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-var-lib-kubelet\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831886 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-lib-modules\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831918 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-system-cni-dir\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831949 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.831924 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-var-lib-kubelet\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbeeb78c-7022-4a4e-8834-767fef6bb920-kubelet-config\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832053 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8136e2a8-4ddc-46fa-b368-c6803750c135-host\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th77c\" (UniqueName: \"kubernetes.io/projected/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-kube-api-access-th77c\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832119 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8136e2a8-4ddc-46fa-b368-c6803750c135-host\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832159 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-device-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-cnibin\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832213 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-log-socket\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832242 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-env-overrides\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.833379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832267 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovnkube-script-lib\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-modprobe-d\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-registration-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gwxw\" (UniqueName: \"kubernetes.io/projected/559de811-5e29-47d5-8dd3-91e74a082d76-kube-api-access-7gwxw\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5zhq\" (UniqueName: \"kubernetes.io/projected/e7cac882-e105-4225-bc58-ee9f8b8f22f4-kube-api-access-r5zhq\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832448 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-device-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832498 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/559de811-5e29-47d5-8dd3-91e74a082d76-registration-dir\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.832573 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-modprobe-d\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.833557 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-etc-tuned\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.834182 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.834055 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-tmp\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.846723 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.846695 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:48:06.846723 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.846727 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:48:06.846924 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.846742 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w4vhf for pod openshift-network-diagnostics/network-check-target-v8wvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:06.846924 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.846908 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf podName:29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:07.346856049 +0000 UTC m=+3.053160231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w4vhf" (UniqueName: "kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf") pod "network-check-target-v8wvp" (UID: "29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:06.847972 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.847946 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqlgg\" (UniqueName: \"kubernetes.io/projected/8136e2a8-4ddc-46fa-b368-c6803750c135-kube-api-access-hqlgg\") pod \"node-ca-k7tz2\" (UID: \"8136e2a8-4ddc-46fa-b368-c6803750c135\") " pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:06.848073 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.847946 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sjq7\" (UniqueName: \"kubernetes.io/projected/1d88de4b-5158-4a97-b796-74a12f088e70-kube-api-access-8sjq7\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:06.848174 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.848152 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th77c\" (UniqueName: \"kubernetes.io/projected/50f0d47a-3ba8-42da-9a53-cfb998bd16e2-kube-api-access-th77c\") pod \"tuned-tv4cv\" (UID: \"50f0d47a-3ba8-42da-9a53-cfb998bd16e2\") " pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:06.848676 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.848656 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gwxw\" (UniqueName: \"kubernetes.io/projected/559de811-5e29-47d5-8dd3-91e74a082d76-kube-api-access-7gwxw\") pod \"aws-ebs-csi-driver-node-mtpr9\" (UID: \"559de811-5e29-47d5-8dd3-91e74a082d76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:06.849271 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.849244 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkcbg\" (UniqueName: \"kubernetes.io/projected/1cd4f44a-0a1e-4710-a68c-be170c527c22-kube-api-access-jkcbg\") pod \"node-resolver-wqrsz\" (UID: \"1cd4f44a-0a1e-4710-a68c-be170c527c22\") " pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:06.933164 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-cnibin\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.933164 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-log-socket\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-env-overrides\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933197 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-cnibin\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovnkube-script-lib\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933252 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5zhq\" (UniqueName: \"kubernetes.io/projected/e7cac882-e105-4225-bc58-ee9f8b8f22f4-kube-api-access-r5zhq\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-hostroot\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933277 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-log-socket\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933295 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-kubelet\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933323 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab-konnectivity-ca\") pod \"konnectivity-agent-j8bc8\" (UID: \"6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab\") " pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-system-cni-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933373 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16a5c16f-7a51-4717-b623-fb1f604add4c-cni-binary-copy\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933358 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-hostroot\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.933406 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933399 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-k8s-cni-cncf-io\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcj7g\" (UniqueName: \"kubernetes.io/projected/16a5c16f-7a51-4717-b623-fb1f604add4c-kube-api-access-vcj7g\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-slash\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-etc-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933532 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-cni-bin\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933580 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933546 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-system-cni-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933612 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-cnibin\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933624 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-kubelet\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933641 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933703 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-var-lib-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933713 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-k8s-cni-cncf-io\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933639 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-var-lib-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933756 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-cni-bin\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-cni-bin\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933789 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-etc-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934039 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933782 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933826 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-env-overrides\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933877 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-cnibin\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933890 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-cni-bin\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933931 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovnkube-config\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab-agent-certs\") pod \"konnectivity-agent-j8bc8\" (UID: \"6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab\") " pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.933985 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-run-netns\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-cni-netd\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7cac882-e105-4225-bc58-ee9f8b8f22f4-host-slash\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-kubelet\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-multus-certs\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934176 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovn-node-metrics-cert\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-os-release\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934229 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-netns\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.934860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934249 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab-konnectivity-ca\") pod \"konnectivity-agent-j8bc8\" (UID: \"6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab\") " pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-systemd-units\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934323 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-systemd-units\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934561 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbeeb78c-7022-4a4e-8834-767fef6bb920-dbus\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934728 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-cni-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934816 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-cni-multus\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934835 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16a5c16f-7a51-4717-b623-fb1f604add4c-cni-binary-copy\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-etc-kubernetes\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-os-release\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934923 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-systemd\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934995 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-conf-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935012 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovnkube-script-lib\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935040 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbeeb78c-7022-4a4e-8834-767fef6bb920-dbus\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935067 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-kubelet\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935111 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-os-release\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935136 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7cac882-e105-4225-bc58-ee9f8b8f22f4-host-slash\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.934904 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-etc-kubernetes\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.935599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935188 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935195 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-cni-netd\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935222 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-cni-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935229 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-var-lib-cni-multus\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935259 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-systemd\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935270 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-daemon-config\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935299 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hqhf\" (UniqueName: \"kubernetes.io/projected/213701d6-76f7-4d6e-8012-6918d482c2c9-kube-api-access-5hqhf\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935310 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-run-netns\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-node-log\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935381 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t79j5\" (UniqueName: \"kubernetes.io/projected/18ff83ff-de0e-418c-bab6-cdbf795df6ce-kube-api-access-t79j5\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935389 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-os-release\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.935419 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935465 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-node-log\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935507 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-conf-dir\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:06.935541 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret podName:cbeeb78c-7022-4a4e-8834-767fef6bb920 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:07.435490038 +0000 UTC m=+3.141794205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret") pod "global-pull-secret-syncer-k5ktc" (UID: "cbeeb78c-7022-4a4e-8834-767fef6bb920") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935635 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-multus-certs\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935711 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-host-run-netns\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.935870 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-host-slash\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936078 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-daemon-config\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936133 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7cac882-e105-4225-bc58-ee9f8b8f22f4-iptables-alerter-script\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936162 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-socket-dir-parent\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-ovn\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936312 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-system-cni-dir\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936414 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbeeb78c-7022-4a4e-8834-767fef6bb920-kubelet-config\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936575 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbeeb78c-7022-4a4e-8834-767fef6bb920-kubelet-config\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:06.936771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936731 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.937035 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936827 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16a5c16f-7a51-4717-b623-fb1f604add4c-multus-socket-dir-parent\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.937035 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936892 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-ovn\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.937035 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936934 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/213701d6-76f7-4d6e-8012-6918d482c2c9-system-cni-dir\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.937035 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.936984 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18ff83ff-de0e-418c-bab6-cdbf795df6ce-run-openvswitch\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.937146 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.937027 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7cac882-e105-4225-bc58-ee9f8b8f22f4-iptables-alerter-script\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.937250 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.937230 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/213701d6-76f7-4d6e-8012-6918d482c2c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:06.939461 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.939368 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovnkube-config\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.939461 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.939421 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18ff83ff-de0e-418c-bab6-cdbf795df6ce-ovn-node-metrics-cert\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.940738 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.940714 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab-agent-certs\") pod \"konnectivity-agent-j8bc8\" (UID: \"6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab\") " pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:06.943465 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.943442 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5zhq\" (UniqueName: \"kubernetes.io/projected/e7cac882-e105-4225-bc58-ee9f8b8f22f4-kube-api-access-r5zhq\") pod \"iptables-alerter-tvzvq\" (UID: \"e7cac882-e105-4225-bc58-ee9f8b8f22f4\") " pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:06.951455 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.951426 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcj7g\" (UniqueName: \"kubernetes.io/projected/16a5c16f-7a51-4717-b623-fb1f604add4c-kube-api-access-vcj7g\") pod \"multus-z22dk\" (UID: \"16a5c16f-7a51-4717-b623-fb1f604add4c\") " pod="openshift-multus/multus-z22dk" Jun 10 12:48:06.952254 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.952234 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t79j5\" (UniqueName: \"kubernetes.io/projected/18ff83ff-de0e-418c-bab6-cdbf795df6ce-kube-api-access-t79j5\") pod \"ovnkube-node-zqwjk\" (UID: \"18ff83ff-de0e-418c-bab6-cdbf795df6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:06.952360 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:06.952322 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hqhf\" (UniqueName: \"kubernetes.io/projected/213701d6-76f7-4d6e-8012-6918d482c2c9-kube-api-access-5hqhf\") pod \"multus-additional-cni-plugins-bz7mt\" (UID: \"213701d6-76f7-4d6e-8012-6918d482c2c9\") " pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:07.020705 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.020625 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wqrsz" Jun 10 12:48:07.026230 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.026208 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k7tz2" Jun 10 12:48:07.041330 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.041279 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" Jun 10 12:48:07.042415 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.042387 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" Jun 10 12:48:07.048420 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.048400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z22dk" Jun 10 12:48:07.055129 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.055112 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" Jun 10 12:48:07.061986 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.061968 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:07.070696 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.070677 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tvzvq" Jun 10 12:48:07.075694 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.075676 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:07.339947 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.339870 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:07.340079 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.340049 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:07.340128 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.340121 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:08.340101237 +0000 UTC m=+4.046405418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:07.440354 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.440315 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:07.440538 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.440372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:07.440538 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.440481 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:07.440538 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.440486 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:48:07.440538 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.440506 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:48:07.440538 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.440519 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w4vhf for pod openshift-network-diagnostics/network-check-target-v8wvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:07.440789 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.440553 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret podName:cbeeb78c-7022-4a4e-8834-767fef6bb920 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:08.440531263 +0000 UTC m=+4.146835431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret") pod "global-pull-secret-syncer-k5ktc" (UID: "cbeeb78c-7022-4a4e-8834-767fef6bb920") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:07.440789 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.440572 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf podName:29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:08.440563048 +0000 UTC m=+4.146867215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w4vhf" (UniqueName: "kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf") pod "network-check-target-v8wvp" (UID: "29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:07.673868 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.673794 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abcc7d2_5450_49f2_9ffb_e7e297e3e9ab.slice/crio-9e6d62243ed49f41245973e36f513bcad6584707c032d19e799b05c465a21bf2 WatchSource:0}: Error finding container 9e6d62243ed49f41245973e36f513bcad6584707c032d19e799b05c465a21bf2: Status 404 returned error can't find the container with id 9e6d62243ed49f41245973e36f513bcad6584707c032d19e799b05c465a21bf2 Jun 10 12:48:07.674856 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.674829 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8136e2a8_4ddc_46fa_b368_c6803750c135.slice/crio-41d6d9cf7c860b08a8a148bdd25f56629637bff62d6df57cf951bddf9faee468 WatchSource:0}: Error finding container 41d6d9cf7c860b08a8a148bdd25f56629637bff62d6df57cf951bddf9faee468: Status 404 returned error can't find the container with id 41d6d9cf7c860b08a8a148bdd25f56629637bff62d6df57cf951bddf9faee468 Jun 10 12:48:07.675750 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.675725 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18ff83ff_de0e_418c_bab6_cdbf795df6ce.slice/crio-09c16b131c6eae91394aa3b6adc2ad2dcf765688407a24f6be9dda4dc2222156 WatchSource:0}: Error finding container 09c16b131c6eae91394aa3b6adc2ad2dcf765688407a24f6be9dda4dc2222156: Status 404 returned error can't find the container with id 09c16b131c6eae91394aa3b6adc2ad2dcf765688407a24f6be9dda4dc2222156 Jun 10 12:48:07.678265 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.677117 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f0d47a_3ba8_42da_9a53_cfb998bd16e2.slice/crio-5ca6840582ab16809f1f575a173ece6d709c3cd7477fe19da15cf6b0641b9542 WatchSource:0}: Error finding container 5ca6840582ab16809f1f575a173ece6d709c3cd7477fe19da15cf6b0641b9542: Status 404 returned error can't find the container with id 5ca6840582ab16809f1f575a173ece6d709c3cd7477fe19da15cf6b0641b9542 Jun 10 12:48:07.679884 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.679860 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559de811_5e29_47d5_8dd3_91e74a082d76.slice/crio-72b022509bef66ef53bad98ed05ac02c92f3e8ac7a673ebae9c806cb4e7c24fb WatchSource:0}: Error finding container 72b022509bef66ef53bad98ed05ac02c92f3e8ac7a673ebae9c806cb4e7c24fb: Status 404 returned error can't find the container with id 72b022509bef66ef53bad98ed05ac02c92f3e8ac7a673ebae9c806cb4e7c24fb Jun 10 12:48:07.680908 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.680887 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7cac882_e105_4225_bc58_ee9f8b8f22f4.slice/crio-f01d0419cc956475df74d59eeb5f3d4c63e06898a166b4b1f909f30b22f0f332 WatchSource:0}: Error finding container f01d0419cc956475df74d59eeb5f3d4c63e06898a166b4b1f909f30b22f0f332: Status 404 returned error can't find the container with id f01d0419cc956475df74d59eeb5f3d4c63e06898a166b4b1f909f30b22f0f332 Jun 10 12:48:07.682920 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.682490 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd4f44a_0a1e_4710_a68c_be170c527c22.slice/crio-3a177e4bfca9a66e3947ed4fa7ed294ab8051c1ea46a82514923c73913673056 WatchSource:0}: Error finding container 3a177e4bfca9a66e3947ed4fa7ed294ab8051c1ea46a82514923c73913673056: Status 404 returned error can't find the container with id 3a177e4bfca9a66e3947ed4fa7ed294ab8051c1ea46a82514923c73913673056 Jun 10 12:48:07.683460 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.683414 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213701d6_76f7_4d6e_8012_6918d482c2c9.slice/crio-31ca7ec826a8925c86f2fb8e2e9cdef9bae81af074ca7b5860e5d52c58622927 WatchSource:0}: Error finding container 31ca7ec826a8925c86f2fb8e2e9cdef9bae81af074ca7b5860e5d52c58622927: Status 404 returned error can't find the container with id 31ca7ec826a8925c86f2fb8e2e9cdef9bae81af074ca7b5860e5d52c58622927 Jun 10 12:48:07.684545 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:07.684523 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a5c16f_7a51_4717_b623_fb1f604add4c.slice/crio-feeabf04cc3f6d5db936e927d4707b80e11696522000f160418c68122a9a7f44 WatchSource:0}: Error finding container feeabf04cc3f6d5db936e927d4707b80e11696522000f160418c68122a9a7f44: Status 404 returned error can't find the container with id feeabf04cc3f6d5db936e927d4707b80e11696522000f160418c68122a9a7f44 Jun 10 12:48:07.774490 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.774440 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 12:43:06 +0000 UTC" deadline="2028-03-16 23:21:21.130967314 +0000 UTC" Jun 10 12:48:07.774490 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.774490 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15490h33m13.356481333s" Jun 10 12:48:07.785690 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.785670 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:07.785780 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:07.785766 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:07.793987 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.793956 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j8bc8" event={"ID":"6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab","Type":"ContainerStarted","Data":"9e6d62243ed49f41245973e36f513bcad6584707c032d19e799b05c465a21bf2"} Jun 10 12:48:07.795310 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.795276 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tvzvq" event={"ID":"e7cac882-e105-4225-bc58-ee9f8b8f22f4","Type":"ContainerStarted","Data":"f01d0419cc956475df74d59eeb5f3d4c63e06898a166b4b1f909f30b22f0f332"} Jun 10 12:48:07.797078 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.797058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" event={"ID":"559de811-5e29-47d5-8dd3-91e74a082d76","Type":"ContainerStarted","Data":"72b022509bef66ef53bad98ed05ac02c92f3e8ac7a673ebae9c806cb4e7c24fb"} Jun 10 12:48:07.798371 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.798348 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" event={"ID":"50f0d47a-3ba8-42da-9a53-cfb998bd16e2","Type":"ContainerStarted","Data":"5ca6840582ab16809f1f575a173ece6d709c3cd7477fe19da15cf6b0641b9542"} Jun 10 12:48:07.800000 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.799918 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"09c16b131c6eae91394aa3b6adc2ad2dcf765688407a24f6be9dda4dc2222156"} Jun 10 12:48:07.803152 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.803129 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" event={"ID":"465107ac2ceb7852c065a227546cfcdf","Type":"ContainerStarted","Data":"d965fffaa72d359ae5186d372654f6c0588a9dd02e779279eb1f99a43be0e550"} Jun 10 12:48:07.804226 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.804205 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z22dk" event={"ID":"16a5c16f-7a51-4717-b623-fb1f604add4c","Type":"ContainerStarted","Data":"feeabf04cc3f6d5db936e927d4707b80e11696522000f160418c68122a9a7f44"} Jun 10 12:48:07.805267 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.805247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wqrsz" event={"ID":"1cd4f44a-0a1e-4710-a68c-be170c527c22","Type":"ContainerStarted","Data":"3a177e4bfca9a66e3947ed4fa7ed294ab8051c1ea46a82514923c73913673056"} Jun 10 12:48:07.806352 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.806328 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" event={"ID":"213701d6-76f7-4d6e-8012-6918d482c2c9","Type":"ContainerStarted","Data":"31ca7ec826a8925c86f2fb8e2e9cdef9bae81af074ca7b5860e5d52c58622927"} Jun 10 12:48:07.807521 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.807500 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k7tz2" event={"ID":"8136e2a8-4ddc-46fa-b368-c6803750c135","Type":"ContainerStarted","Data":"41d6d9cf7c860b08a8a148bdd25f56629637bff62d6df57cf951bddf9faee468"} Jun 10 12:48:07.817696 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:07.817642 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-124.ec2.internal" podStartSLOduration=1.8176278510000001 podStartE2EDuration="1.817627851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:07.817495444 +0000 UTC m=+3.523799647" watchObservedRunningTime="2026-06-10 12:48:07.817627851 +0000 UTC m=+3.523932041" Jun 10 12:48:08.349295 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.349208 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:08.349464 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.349384 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:08.349464 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.349444 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:10.34942604 +0000 UTC m=+6.055730212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:08.450569 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.449821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:08.450569 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.449934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:08.450569 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.450059 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:08.450569 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.450124 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret podName:cbeeb78c-7022-4a4e-8834-767fef6bb920 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:10.450103689 +0000 UTC m=+6.156407861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret") pod "global-pull-secret-syncer-k5ktc" (UID: "cbeeb78c-7022-4a4e-8834-767fef6bb920") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:08.450569 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.450555 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:48:08.450569 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.450576 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:48:08.451035 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.450589 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w4vhf for pod openshift-network-diagnostics/network-check-target-v8wvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:08.451035 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.450633 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf podName:29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:10.450617939 +0000 UTC m=+6.156922111 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w4vhf" (UniqueName: "kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf") pod "network-check-target-v8wvp" (UID: "29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:08.774901 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.774779 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 12:43:06 +0000 UTC" deadline="2027-11-03 15:33:45.20145784 +0000 UTC" Jun 10 12:48:08.774901 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.774825 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12266h45m36.426637582s" Jun 10 12:48:08.786763 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.786737 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:08.786939 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.786898 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:08.787431 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.787383 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:08.787535 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:08.787476 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:08.838152 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.837423 2577 generic.go:356] "Generic (PLEG): container finished" podID="a301da613e9794c1472f3518634c2dc6" containerID="52fee4aecf4a013ea06b40910e45705b51b249b93d804885b511b34e74ae4430" exitCode=0 Jun 10 12:48:08.838152 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:08.837889 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" event={"ID":"a301da613e9794c1472f3518634c2dc6","Type":"ContainerDied","Data":"52fee4aecf4a013ea06b40910e45705b51b249b93d804885b511b34e74ae4430"} Jun 10 12:48:09.786415 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:09.786379 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:09.786887 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:09.786526 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:09.868149 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:09.867398 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" event={"ID":"a301da613e9794c1472f3518634c2dc6","Type":"ContainerStarted","Data":"60beb017df692c206bd6961dc1172284c03dc9647ea5a8c525fbfc3317d0918c"} Jun 10 12:48:10.366758 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:10.366720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:10.366975 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.366951 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:10.367044 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.367017 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:14.366998708 +0000 UTC m=+10.073302881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:10.467524 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:10.467482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:10.467711 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:10.467571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:10.467864 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.467841 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:10.467992 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.467921 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret podName:cbeeb78c-7022-4a4e-8834-767fef6bb920 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:14.467902341 +0000 UTC m=+10.174206515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret") pod "global-pull-secret-syncer-k5ktc" (UID: "cbeeb78c-7022-4a4e-8834-767fef6bb920") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:10.468384 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.468366 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:48:10.468467 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.468388 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:48:10.468467 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.468401 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w4vhf for pod openshift-network-diagnostics/network-check-target-v8wvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:10.468467 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.468446 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf podName:29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:14.468430244 +0000 UTC m=+10.174734414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w4vhf" (UniqueName: "kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf") pod "network-check-target-v8wvp" (UID: "29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:10.786942 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:10.786833 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:10.787419 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.786970 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:10.787419 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:10.787305 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:10.787419 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:10.787390 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:11.786448 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:11.786203 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:11.786448 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:11.786357 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:12.786603 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:12.786388 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:12.786603 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:12.786516 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:12.786603 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:12.786567 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:12.787172 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:12.786680 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:13.786235 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:13.786201 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:13.786435 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:13.786353 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:14.401140 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:14.401101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:14.401627 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.401276 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:14.401627 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.401339 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:22.401319674 +0000 UTC m=+18.107623847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:14.502226 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:14.502179 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:14.502423 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:14.502256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:14.502423 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.502367 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:14.502423 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.502400 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:48:14.502423 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.502415 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:48:14.502624 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.502429 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w4vhf for pod openshift-network-diagnostics/network-check-target-v8wvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:14.502624 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.502444 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret podName:cbeeb78c-7022-4a4e-8834-767fef6bb920 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:22.502422166 +0000 UTC m=+18.208726337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret") pod "global-pull-secret-syncer-k5ktc" (UID: "cbeeb78c-7022-4a4e-8834-767fef6bb920") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:14.502624 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.502478 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf podName:29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:22.50246242 +0000 UTC m=+18.208766602 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w4vhf" (UniqueName: "kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf") pod "network-check-target-v8wvp" (UID: "29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:14.788026 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:14.787922 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:14.788173 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.788045 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:14.789718 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:14.789696 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:14.790010 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:14.789989 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:15.787067 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:15.786557 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:15.787067 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:15.786694 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:16.786730 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:16.786682 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:16.786922 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:16.786827 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:16.786922 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:16.786853 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:16.787018 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:16.786947 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:17.786073 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:17.786039 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:17.786587 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:17.786180 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:18.786028 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:18.785989 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:18.786201 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:18.786119 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:18.786201 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:18.786157 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:18.786580 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:18.786224 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:19.786675 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:19.786648 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:19.787162 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:19.786762 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:20.786374 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:20.786339 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:20.786523 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:20.786345 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:20.786523 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:20.786478 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:20.786640 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:20.786620 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:21.786357 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:21.786317 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:21.786833 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:21.786455 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:22.462824 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:22.462771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:22.463019 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.462947 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:22.463075 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.463030 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:38.463005911 +0000 UTC m=+34.169310078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:22.563893 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:22.563857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:22.564072 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:22.563929 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:22.564072 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.564037 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:22.564072 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.564050 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:48:22.564072 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.564068 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:48:22.564246 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.564080 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w4vhf for pod openshift-network-diagnostics/network-check-target-v8wvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:22.564246 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.564117 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret podName:cbeeb78c-7022-4a4e-8834-767fef6bb920 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:38.564096413 +0000 UTC m=+34.270400600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret") pod "global-pull-secret-syncer-k5ktc" (UID: "cbeeb78c-7022-4a4e-8834-767fef6bb920") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:22.564246 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.564138 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf podName:29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9 nodeName:}" failed. No retries permitted until 2026-06-10 12:48:38.564127829 +0000 UTC m=+34.270432003 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w4vhf" (UniqueName: "kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf") pod "network-check-target-v8wvp" (UID: "29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:22.786391 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:22.786288 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:22.786846 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:22.786288 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:22.786846 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.786420 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:22.786846 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:22.786541 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:23.785877 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:23.785843 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:23.786035 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:23.785960 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:24.787444 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:24.787404 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:24.787936 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:24.787507 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:24.787936 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:24.787544 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:24.787936 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:24.787579 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:25.786235 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:25.786204 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:25.786420 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:25.786325 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:26.786285 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.786262 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:26.787056 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:26.786365 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:26.787056 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.787026 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:26.787178 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:26.787105 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:26.904210 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.904173 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" event={"ID":"559de811-5e29-47d5-8dd3-91e74a082d76","Type":"ContainerStarted","Data":"c683799a9bbc18653f28291d7f2f962838b0235ea67944eae6a7e33892575e60"} Jun 10 12:48:26.905665 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.905631 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" event={"ID":"50f0d47a-3ba8-42da-9a53-cfb998bd16e2","Type":"ContainerStarted","Data":"7ee6333050aa95905676d765ddbb33c802a44d6195dbc6f57ffb8cc456259f62"} Jun 10 12:48:26.907227 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.907205 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:48:26.907588 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.907568 2577 generic.go:356] "Generic (PLEG): container finished" podID="18ff83ff-de0e-418c-bab6-cdbf795df6ce" containerID="2fb24598f009bcaefdd4c5d29e9961991676bb9a7abdce1a57ef0e791f8c0a7c" exitCode=1 Jun 10 12:48:26.907676 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.907636 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerDied","Data":"2fb24598f009bcaefdd4c5d29e9961991676bb9a7abdce1a57ef0e791f8c0a7c"} Jun 10 12:48:26.907732 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.907706 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"4a2329bad243bfc80d22bc1fb3dc708395fefd883d4f69be7bf4335b6cc56686"} Jun 10 12:48:26.909195 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.909168 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z22dk" event={"ID":"16a5c16f-7a51-4717-b623-fb1f604add4c","Type":"ContainerStarted","Data":"c954619a3206d793112729c56818e2020b62a5a8c8a6be617d8ae12687b88f22"} Jun 10 12:48:26.910851 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.910825 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wqrsz" event={"ID":"1cd4f44a-0a1e-4710-a68c-be170c527c22","Type":"ContainerStarted","Data":"f7cc5dc1a9f9968eaf1676632963dbd2030220f3db1cb68c4810e6e383d40309"} Jun 10 12:48:26.912151 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.912125 2577 generic.go:356] "Generic (PLEG): container finished" podID="213701d6-76f7-4d6e-8012-6918d482c2c9" containerID="0f480c2bc6ef805d789ea26b41870a251dec25a4d83589dc12697d3d16cbf8cc" exitCode=0 Jun 10 12:48:26.912275 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.912192 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" event={"ID":"213701d6-76f7-4d6e-8012-6918d482c2c9","Type":"ContainerDied","Data":"0f480c2bc6ef805d789ea26b41870a251dec25a4d83589dc12697d3d16cbf8cc"} Jun 10 12:48:26.913654 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.913633 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k7tz2" event={"ID":"8136e2a8-4ddc-46fa-b368-c6803750c135","Type":"ContainerStarted","Data":"7a54faef3fdb6b2271bd8c266d4262c50788caba3421e2e102f9b97e6d9cae5f"} Jun 10 12:48:26.917056 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.917023 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j8bc8" event={"ID":"6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab","Type":"ContainerStarted","Data":"98884969f1f94ef6e72137afc17c5aa992d6b30abc3ab33fe663d818dded0242"} Jun 10 12:48:26.927953 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.927911 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tv4cv" podStartSLOduration=4.116496012 podStartE2EDuration="22.927898112s" totalImagesPullingTime="18.8114021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:26.926959888 +0000 UTC m=+22.633264077" watchObservedRunningTime="2026-06-10 12:48:26.927898112 +0000 UTC m=+22.634202298" Jun 10 12:48:26.928183 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.928160 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-124.ec2.internal" podStartSLOduration=20.928153797 podStartE2EDuration="20.928153797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:09.889780312 +0000 UTC m=+5.596084502" watchObservedRunningTime="2026-06-10 12:48:26.928153797 +0000 UTC m=+22.634457985" Jun 10 12:48:26.950971 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.950176 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j8bc8" podStartSLOduration=11.577913967 podStartE2EDuration="21.950157243s" totalImagesPullingTime="10.372243276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:26.949954861 +0000 UTC m=+22.656259051" watchObservedRunningTime="2026-06-10 12:48:26.950157243 +0000 UTC m=+22.656461433" Jun 10 12:48:26.967877 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:26.967834 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wqrsz" podStartSLOduration=4.163092989 podStartE2EDuration="22.967795242s" totalImagesPullingTime="18.804702253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:26.967230685 +0000 UTC m=+22.673534877" watchObservedRunningTime="2026-06-10 12:48:26.967795242 +0000 UTC m=+22.674099431" Jun 10 12:48:27.009237 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.009011 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k7tz2" podStartSLOduration=4.197163964 podStartE2EDuration="23.00899008s" totalImagesPullingTime="18.811826116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:27.008473932 +0000 UTC m=+22.714778123" watchObservedRunningTime="2026-06-10 12:48:27.00899008 +0000 UTC m=+22.715294273" Jun 10 12:48:27.029666 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.029622 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z22dk" podStartSLOduration=4.174535687 podStartE2EDuration="23.029608505s" totalImagesPullingTime="18.855072818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:27.029040387 +0000 UTC m=+22.735344575" watchObservedRunningTime="2026-06-10 12:48:27.029608505 +0000 UTC m=+22.735912694" Jun 10 12:48:27.786571 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.786541 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:27.787204 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:27.786678 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:27.922565 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.922543 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:48:27.923112 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.922999 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"3f687fa615a17133703dbc03622352418510ebe2c2ece62123c8a59936e638bd"} Jun 10 12:48:27.923112 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.923090 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"a66942a90fd129e67f39cdf9a7449089a980d644cc03a283ce7af67467e163fa"} Jun 10 12:48:27.923379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.923353 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"d314d194a24b110f98e3c8070b90dc96d39f32396f8b38efcc893cc511269984"} Jun 10 12:48:27.923508 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.923421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"67a264a70ff95f871b289dfc5708e2ecf8df76f0e001fbfdc5097880bccd0bbc"} Jun 10 12:48:27.965820 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:27.965774 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 10 12:48:28.786640 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:28.786537 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-10T12:48:27.965797709Z","UUID":"6e28d788-16c1-4c35-8b01-d9e33a12d58a","Handler":null,"Name":"","Endpoint":""} Jun 10 12:48:28.786640 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:28.786628 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:28.787188 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:28.786745 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:28.787188 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:28.786747 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:28.787188 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:28.786844 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:28.790829 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:28.788402 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 10 12:48:28.790829 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:28.788449 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 10 12:48:28.927393 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:28.927316 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tvzvq" event={"ID":"e7cac882-e105-4225-bc58-ee9f8b8f22f4","Type":"ContainerStarted","Data":"0b3e3a99580438b852f27d01bc20017602719dd4ef1c69acbc2544f09729434b"} Jun 10 12:48:28.929321 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:28.929298 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" event={"ID":"559de811-5e29-47d5-8dd3-91e74a082d76","Type":"ContainerStarted","Data":"82e5086b9b15ce98f84e8d281b228af943a1d2b69592f86f3543c4a7f110c45b"} Jun 10 12:48:29.786211 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:29.786174 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:29.786413 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:29.786310 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:29.934172 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:29.934136 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" event={"ID":"559de811-5e29-47d5-8dd3-91e74a082d76","Type":"ContainerStarted","Data":"59668e666e24e538452d3728a942adf1c72fe754f1281b7b76c25dfbb5818247"} Jun 10 12:48:29.937584 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:29.937558 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:48:29.938051 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:29.938026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"6d0888bbf6ab5184429be07c59a485181c41f9d5f260279893e18ede082a8255"} Jun 10 12:48:29.957482 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:29.957432 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtpr9" podStartSLOduration=5.105923327 podStartE2EDuration="25.957411297s" totalImagesPullingTime="20.85148797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:29.956234229 +0000 UTC m=+25.662538448" watchObservedRunningTime="2026-06-10 12:48:29.957411297 +0000 UTC m=+25.663715489" Jun 10 12:48:29.957987 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:29.957951 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tvzvq" podStartSLOduration=7.151459957 podStartE2EDuration="25.957938869s" totalImagesPullingTime="18.806478912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:28.943229991 +0000 UTC m=+24.649534180" watchObservedRunningTime="2026-06-10 12:48:29.957938869 +0000 UTC m=+25.664243105" Jun 10 12:48:30.351575 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:30.351523 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:30.352181 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:30.352162 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:30.786045 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:30.786009 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:30.786045 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:30.786049 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:30.786272 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:30.786156 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:30.786326 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:30.786283 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:30.939784 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:30.939753 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:30.940351 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:30.940096 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j8bc8" Jun 10 12:48:31.786898 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.786686 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:31.787133 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:31.786977 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:31.945476 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.945452 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:48:31.946075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.945835 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"409da5fc3dfc9f2b5e5504708b6898cc0a33d1e8a9fccbf8147e0c1220138bf5"} Jun 10 12:48:31.946120 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.946095 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:31.946183 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.946151 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:31.946183 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.946182 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:31.946430 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.946407 2577 scope.go:117] "RemoveContainer" containerID="2fb24598f009bcaefdd4c5d29e9961991676bb9a7abdce1a57ef0e791f8c0a7c" Jun 10 12:48:31.947917 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.947884 2577 generic.go:356] "Generic (PLEG): container finished" podID="213701d6-76f7-4d6e-8012-6918d482c2c9" containerID="c1f9575e40aba2addbaa61766d91255e603a30ee177b9220d98f0f90353174a9" exitCode=0 Jun 10 12:48:31.948026 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.947971 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" event={"ID":"213701d6-76f7-4d6e-8012-6918d482c2c9","Type":"ContainerDied","Data":"c1f9575e40aba2addbaa61766d91255e603a30ee177b9220d98f0f90353174a9"} Jun 10 12:48:31.963486 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.963462 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:31.963876 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:31.963860 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:48:32.787058 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:32.787020 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:32.787278 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:32.787029 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:32.787278 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:32.787163 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:32.787278 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:32.787196 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:32.953065 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:32.953035 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:48:32.953512 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:32.953410 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" event={"ID":"18ff83ff-de0e-418c-bab6-cdbf795df6ce","Type":"ContainerStarted","Data":"6a0ec3fd2e61f2e0e0aafac9161f68c2ca1f9b0587a0e40f7d30d97a92fa60e0"} Jun 10 12:48:32.982504 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:32.982455 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" podStartSLOduration=10.129244896 podStartE2EDuration="28.982440999s" totalImagesPullingTime="18.853196103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:32.980835499 +0000 UTC m=+28.687139687" watchObservedRunningTime="2026-06-10 12:48:32.982440999 +0000 UTC m=+28.688745188" Jun 10 12:48:33.732606 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:33.732352 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v8wvp"] Jun 10 12:48:33.732747 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:33.732676 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:33.732820 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:33.732786 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:33.735411 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:33.735385 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gcbmf"] Jun 10 12:48:33.735543 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:33.735517 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:33.735657 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:33.735635 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:33.736169 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:33.736147 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k5ktc"] Jun 10 12:48:33.736241 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:33.736234 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:33.736341 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:33.736321 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:33.957767 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:33.957740 2577 generic.go:356] "Generic (PLEG): container finished" podID="213701d6-76f7-4d6e-8012-6918d482c2c9" containerID="02c28aef9e505d6a09d805e197eb7b9282216cf3459483109df86811097fdf44" exitCode=0 Jun 10 12:48:33.958176 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:33.957794 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" event={"ID":"213701d6-76f7-4d6e-8012-6918d482c2c9","Type":"ContainerDied","Data":"02c28aef9e505d6a09d805e197eb7b9282216cf3459483109df86811097fdf44"} Jun 10 12:48:35.785975 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:35.785879 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:35.786710 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:35.785883 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:35.786710 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:35.785995 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:35.786710 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:35.786076 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:35.786710 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:35.785894 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:35.786710 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:35.786157 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:35.965472 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:35.965431 2577 generic.go:356] "Generic (PLEG): container finished" podID="213701d6-76f7-4d6e-8012-6918d482c2c9" containerID="dcfeeba9a9e07ff80450b8033ee25880e7f0b2f699a3a1252d74b21dea8b5c16" exitCode=0 Jun 10 12:48:35.965640 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:35.965507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" event={"ID":"213701d6-76f7-4d6e-8012-6918d482c2c9","Type":"ContainerDied","Data":"dcfeeba9a9e07ff80450b8033ee25880e7f0b2f699a3a1252d74b21dea8b5c16"} Jun 10 12:48:37.786660 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:37.786626 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:37.787237 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:37.786742 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5ktc" podUID="cbeeb78c-7022-4a4e-8834-767fef6bb920" Jun 10 12:48:37.787237 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:37.786626 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:37.787237 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:37.786626 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:37.787237 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:37.786855 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:48:37.787237 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:37.786923 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8wvp" podUID="29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9" Jun 10 12:48:38.476787 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:38.476748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:38.477043 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:38.476908 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:38.477043 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:38.476974 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:10.476957454 +0000 UTC m=+66.183261621 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:48:38.577586 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:38.577552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:38.577777 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:38.577610 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:38.577777 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:38.577706 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:38.577777 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:38.577740 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:48:38.577777 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:38.577769 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:48:38.577777 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:38.577776 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret podName:cbeeb78c-7022-4a4e-8834-767fef6bb920 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:10.577751981 +0000 UTC m=+66.284056148 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret") pod "global-pull-secret-syncer-k5ktc" (UID: "cbeeb78c-7022-4a4e-8834-767fef6bb920") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:48:38.578036 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:38.577783 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w4vhf for pod openshift-network-diagnostics/network-check-target-v8wvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:38.578036 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:38.577877 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf podName:29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:10.577834053 +0000 UTC m=+66.284138237 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w4vhf" (UniqueName: "kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf") pod "network-check-target-v8wvp" (UID: "29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:48:39.578930 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.578852 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-124.ec2.internal" event="NodeReady" Jun 10 12:48:39.579345 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.579019 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 10 12:48:39.635530 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.635494 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sbvn9"] Jun 10 12:48:39.662057 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.662014 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v78tb"] Jun 10 12:48:39.662221 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.662164 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sbvn9" Jun 10 12:48:39.666849 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.666406 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 10 12:48:39.666849 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.666590 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xqtln\"" Jun 10 12:48:39.667069 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.666931 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 10 12:48:39.667127 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.667097 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 10 12:48:39.677984 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.677949 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sbvn9"] Jun 10 12:48:39.677984 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.677985 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v78tb"] Jun 10 12:48:39.678169 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.678099 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.680739 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.680719 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 10 12:48:39.680872 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.680719 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 10 12:48:39.680872 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.680843 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s86xl\"" Jun 10 12:48:39.786011 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.785982 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:48:39.786011 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.786017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1-cert\") pod \"ingress-canary-sbvn9\" (UID: \"513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1\") " pod="openshift-ingress-canary/ingress-canary-sbvn9" Jun 10 12:48:39.786252 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.786049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr77v\" (UniqueName: \"kubernetes.io/projected/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-kube-api-access-rr77v\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.786252 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.786088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmxkk\" (UniqueName: \"kubernetes.io/projected/513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1-kube-api-access-gmxkk\") pod \"ingress-canary-sbvn9\" (UID: \"513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1\") " pod="openshift-ingress-canary/ingress-canary-sbvn9" Jun 10 12:48:39.786252 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.786109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:48:39.786252 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.786112 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-config-volume\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.786252 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.786149 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:48:39.786252 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.786196 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-tmp-dir\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.786543 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.786256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.788851 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.788824 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 12:48:39.788993 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.788824 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 12:48:39.788993 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.788988 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 12:48:39.789156 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.789137 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g75kp\"" Jun 10 12:48:39.789301 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.789206 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7d5dq\"" Jun 10 12:48:39.789301 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.789241 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 12:48:39.887107 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.887073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmxkk\" (UniqueName: \"kubernetes.io/projected/513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1-kube-api-access-gmxkk\") pod \"ingress-canary-sbvn9\" (UID: \"513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1\") " pod="openshift-ingress-canary/ingress-canary-sbvn9" Jun 10 12:48:39.887395 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.887113 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-config-volume\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.887395 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.887303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-tmp-dir\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.887395 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.887374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.887563 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.887462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1-cert\") pod \"ingress-canary-sbvn9\" (UID: \"513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1\") " pod="openshift-ingress-canary/ingress-canary-sbvn9" Jun 10 12:48:39.887563 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.887487 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr77v\" (UniqueName: \"kubernetes.io/projected/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-kube-api-access-rr77v\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.887563 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:39.887555 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:48:39.887710 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:39.887665 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:48:40.387645134 +0000 UTC m=+36.093949316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:48:39.887710 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.887684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-tmp-dir\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.887871 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.887842 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-config-volume\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.892639 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.892454 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1-cert\") pod \"ingress-canary-sbvn9\" (UID: \"513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1\") " pod="openshift-ingress-canary/ingress-canary-sbvn9" Jun 10 12:48:39.899868 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.899840 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr77v\" (UniqueName: \"kubernetes.io/projected/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-kube-api-access-rr77v\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:39.900009 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.899989 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmxkk\" (UniqueName: \"kubernetes.io/projected/513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1-kube-api-access-gmxkk\") pod \"ingress-canary-sbvn9\" (UID: \"513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1\") " pod="openshift-ingress-canary/ingress-canary-sbvn9" Jun 10 12:48:39.973980 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:39.973945 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sbvn9" Jun 10 12:48:40.177083 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:40.176968 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sbvn9"] Jun 10 12:48:40.181117 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:48:40.181082 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513a89d3_0aa0_4f3a_8acb_ce4f074ac9f1.slice/crio-f90caa509be47a735ec0266b06def28747d7bab03e6ded37f95a3a7c7acd78c5 WatchSource:0}: Error finding container f90caa509be47a735ec0266b06def28747d7bab03e6ded37f95a3a7c7acd78c5: Status 404 returned error can't find the container with id f90caa509be47a735ec0266b06def28747d7bab03e6ded37f95a3a7c7acd78c5 Jun 10 12:48:40.391880 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:40.391856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:40.391996 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:40.391978 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:48:40.392035 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:40.392027 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:48:41.392014596 +0000 UTC m=+37.098318762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:48:40.981504 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:40.981466 2577 generic.go:356] "Generic (PLEG): container finished" podID="213701d6-76f7-4d6e-8012-6918d482c2c9" containerID="1a737ce280381133142e434375c6a495f11bfa0eeab0edd77f4512fcd9fb2f2b" exitCode=0 Jun 10 12:48:40.981986 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:40.981564 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" event={"ID":"213701d6-76f7-4d6e-8012-6918d482c2c9","Type":"ContainerDied","Data":"1a737ce280381133142e434375c6a495f11bfa0eeab0edd77f4512fcd9fb2f2b"} Jun 10 12:48:40.983001 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:40.982978 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sbvn9" event={"ID":"513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1","Type":"ContainerStarted","Data":"f90caa509be47a735ec0266b06def28747d7bab03e6ded37f95a3a7c7acd78c5"} Jun 10 12:48:41.399671 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:41.399606 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:41.399884 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:41.399781 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:48:41.399884 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:41.399880 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:48:43.399858586 +0000 UTC m=+39.106162770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:48:41.988474 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:41.988296 2577 generic.go:356] "Generic (PLEG): container finished" podID="213701d6-76f7-4d6e-8012-6918d482c2c9" containerID="0dcbc7e87e8ba21918c61445baf9481d4bb23c957d8d4bd250c59977f6ac35a0" exitCode=0 Jun 10 12:48:41.988920 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:41.988383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" event={"ID":"213701d6-76f7-4d6e-8012-6918d482c2c9","Type":"ContainerDied","Data":"0dcbc7e87e8ba21918c61445baf9481d4bb23c957d8d4bd250c59977f6ac35a0"} Jun 10 12:48:41.990169 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:41.990142 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sbvn9" event={"ID":"513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1","Type":"ContainerStarted","Data":"07f075fcf56971a0ea62547d5a0da2c3779247edd8c6fc836681cdc5eaca8427"} Jun 10 12:48:42.029913 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:42.029864 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sbvn9" podStartSLOduration=1.366380017 podStartE2EDuration="3.029843343s" totalImagesPullingTime="1.663463326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:42.028928903 +0000 UTC m=+37.735233090" watchObservedRunningTime="2026-06-10 12:48:42.029843343 +0000 UTC m=+37.736147532" Jun 10 12:48:42.995570 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:42.995535 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" event={"ID":"213701d6-76f7-4d6e-8012-6918d482c2c9","Type":"ContainerStarted","Data":"6012b95fa4ee7c2289e7bbe96a4e217e88e0e4061cde9a18da0f49bdbffa5143"} Jun 10 12:48:43.021971 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:43.021925 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bz7mt" podStartSLOduration=9.320227939 podStartE2EDuration="39.021908754s" totalImagesPullingTime="29.701680815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:48:43.021797319 +0000 UTC m=+38.728101520" watchObservedRunningTime="2026-06-10 12:48:43.021908754 +0000 UTC m=+38.728212944" Jun 10 12:48:43.416254 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:43.416213 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:43.416426 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:43.416334 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:48:43.416426 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:43.416384 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:48:47.416371535 +0000 UTC m=+43.122675701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:48:47.445133 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:47.445089 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:47.445510 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:47.445252 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:48:47.445510 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:47.445316 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:48:55.44530016 +0000 UTC m=+51.151604327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:48:55.495969 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:48:55.495928 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:48:55.496502 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:55.496049 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:48:55.496502 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:48:55.496101 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:49:11.496087761 +0000 UTC m=+67.202391928 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:49:03.977928 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:03.977900 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqwjk" Jun 10 12:49:10.502126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.502067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:49:10.504956 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.504938 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 12:49:10.512464 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:49:10.512445 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:49:10.512521 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:49:10.512503 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:50:14.51248792 +0000 UTC m=+130.218792087 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : secret "metrics-daemon-secret" not found Jun 10 12:49:10.602842 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.602778 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:49:10.603014 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.602873 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:49:10.605752 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.605733 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 12:49:10.605817 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.605733 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 12:49:10.616220 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.616204 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 12:49:10.617183 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.617164 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbeeb78c-7022-4a4e-8834-767fef6bb920-original-pull-secret\") pod \"global-pull-secret-syncer-k5ktc\" (UID: \"cbeeb78c-7022-4a4e-8834-767fef6bb920\") " pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:49:10.626227 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.626206 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4vhf\" (UniqueName: \"kubernetes.io/projected/29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9-kube-api-access-w4vhf\") pod \"network-check-target-v8wvp\" (UID: \"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9\") " pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:49:10.702210 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.702179 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g75kp\"" Jun 10 12:49:10.706284 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.706266 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5ktc" Jun 10 12:49:10.709755 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.709742 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:49:10.855883 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.855853 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v8wvp"] Jun 10 12:49:10.859798 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:49:10.859773 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f6e3d9_d93f_4a41_b2c1_5e750a41a5e9.slice/crio-a0e13791281ebef2eaee07cd8d1e8f11ae6edac3c8ea311bac0f257b019481ed WatchSource:0}: Error finding container a0e13791281ebef2eaee07cd8d1e8f11ae6edac3c8ea311bac0f257b019481ed: Status 404 returned error can't find the container with id a0e13791281ebef2eaee07cd8d1e8f11ae6edac3c8ea311bac0f257b019481ed Jun 10 12:49:10.872947 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:10.872923 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k5ktc"] Jun 10 12:49:10.875518 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:49:10.875498 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbeeb78c_7022_4a4e_8834_767fef6bb920.slice/crio-ce9e16f280c3834cfe7349113e197319528f56da86e4d912995786563bdf268c WatchSource:0}: Error finding container ce9e16f280c3834cfe7349113e197319528f56da86e4d912995786563bdf268c: Status 404 returned error can't find the container with id ce9e16f280c3834cfe7349113e197319528f56da86e4d912995786563bdf268c Jun 10 12:49:11.064277 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:11.064185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v8wvp" event={"ID":"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9","Type":"ContainerStarted","Data":"a0e13791281ebef2eaee07cd8d1e8f11ae6edac3c8ea311bac0f257b019481ed"} Jun 10 12:49:11.065191 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:11.065170 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k5ktc" event={"ID":"cbeeb78c-7022-4a4e-8834-767fef6bb920","Type":"ContainerStarted","Data":"ce9e16f280c3834cfe7349113e197319528f56da86e4d912995786563bdf268c"} Jun 10 12:49:11.509335 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:11.509299 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:49:11.509788 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:49:11.509449 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:49:11.509788 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:49:11.509514 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:49:43.509493818 +0000 UTC m=+99.215797988 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:49:16.080185 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:16.080143 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v8wvp" event={"ID":"29f6e3d9-d93f-4a41-b2c1-5e750a41a5e9","Type":"ContainerStarted","Data":"241b54dfde197ed5dcaa063a7ddc31425abf853ae22cf144cd4e3bb9db6e2e9c"} Jun 10 12:49:16.080627 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:16.080270 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:49:16.081627 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:16.081606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k5ktc" event={"ID":"cbeeb78c-7022-4a4e-8834-767fef6bb920","Type":"ContainerStarted","Data":"fd93ddf001776f03c4f163acb3bf50ad7bad6e3f87aea9aa56b77a0daeaf8016"} Jun 10 12:49:16.097388 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:16.097326 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v8wvp" podStartSLOduration=67.412619343 podStartE2EDuration="1m12.09730913s" totalImagesPullingTime="4.684689787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:16.096940772 +0000 UTC m=+71.803244961" watchObservedRunningTime="2026-06-10 12:49:16.09730913 +0000 UTC m=+71.803613322" Jun 10 12:49:16.112304 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:16.112262 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k5ktc" podStartSLOduration=66.442722342 podStartE2EDuration="1m11.112247927s" totalImagesPullingTime="4.669525585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:16.11116309 +0000 UTC m=+71.817467279" watchObservedRunningTime="2026-06-10 12:49:16.112247927 +0000 UTC m=+71.818552157" Jun 10 12:49:17.705262 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.705226 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7"] Jun 10 12:49:17.710012 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.709986 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.712686 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.712665 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 10 12:49:17.712851 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.712835 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 10 12:49:17.713824 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.713790 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 10 12:49:17.713926 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.713844 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 10 12:49:17.721619 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.721598 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7"] Jun 10 12:49:17.724738 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.724714 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb"] Jun 10 12:49:17.727554 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.727537 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.730974 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.730950 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 10 12:49:17.731231 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.731216 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 10 12:49:17.731273 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.731217 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 10 12:49:17.731273 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.731245 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 10 12:49:17.739542 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.739524 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb"] Jun 10 12:49:17.855439 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855395 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.855620 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855459 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ff89facf-123c-4f6f-8fb8-d13eff6410b9-klusterlet-config\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.855620 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855483 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.855620 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a2eb0f-40e5-4697-943e-299c4facfcba-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.855620 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855528 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8nx4\" (UniqueName: \"kubernetes.io/projected/ff89facf-123c-4f6f-8fb8-d13eff6410b9-kube-api-access-v8nx4\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.855620 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-ca\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.855620 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855607 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-hub\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.855833 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpq9j\" (UniqueName: \"kubernetes.io/projected/f0a2eb0f-40e5-4697-943e-299c4facfcba-kube-api-access-dpq9j\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.855833 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.855707 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff89facf-123c-4f6f-8fb8-d13eff6410b9-tmp\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.956903 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.956777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpq9j\" (UniqueName: \"kubernetes.io/projected/f0a2eb0f-40e5-4697-943e-299c4facfcba-kube-api-access-dpq9j\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.956903 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.956847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff89facf-123c-4f6f-8fb8-d13eff6410b9-tmp\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.956903 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.956863 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.956903 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.956891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ff89facf-123c-4f6f-8fb8-d13eff6410b9-klusterlet-config\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.956903 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.956907 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.957254 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.956924 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a2eb0f-40e5-4697-943e-299c4facfcba-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.957254 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.956948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8nx4\" (UniqueName: \"kubernetes.io/projected/ff89facf-123c-4f6f-8fb8-d13eff6410b9-kube-api-access-v8nx4\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.957254 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.956966 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-ca\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.957254 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.957174 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-hub\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.957509 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.957310 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff89facf-123c-4f6f-8fb8-d13eff6410b9-tmp\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.957997 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.957968 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a2eb0f-40e5-4697-943e-299c4facfcba-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.959490 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.959460 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-ca\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.959647 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.959624 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.959727 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.959705 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ff89facf-123c-4f6f-8fb8-d13eff6410b9-klusterlet-config\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.959860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.959840 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.959973 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.959958 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f0a2eb0f-40e5-4697-943e-299c4facfcba-hub\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:17.965384 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.965361 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8nx4\" (UniqueName: \"kubernetes.io/projected/ff89facf-123c-4f6f-8fb8-d13eff6410b9-kube-api-access-v8nx4\") pod \"klusterlet-addon-workmgr-f8bcd54-fkph7\" (UID: \"ff89facf-123c-4f6f-8fb8-d13eff6410b9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:17.966028 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:17.965998 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpq9j\" (UniqueName: \"kubernetes.io/projected/f0a2eb0f-40e5-4697-943e-299c4facfcba-kube-api-access-dpq9j\") pod \"cluster-proxy-proxy-agent-58b7b76b55-m4gxb\" (UID: \"f0a2eb0f-40e5-4697-943e-299c4facfcba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:18.018871 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:18.018840 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:18.049789 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:18.049765 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:49:18.147725 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:18.147698 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7"] Jun 10 12:49:18.162606 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:49:18.162579 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff89facf_123c_4f6f_8fb8_d13eff6410b9.slice/crio-c624507447deb2c1cec11f44d4455f6c389b34a7788ccd7b05c6ab72ae8641e9 WatchSource:0}: Error finding container c624507447deb2c1cec11f44d4455f6c389b34a7788ccd7b05c6ab72ae8641e9: Status 404 returned error can't find the container with id c624507447deb2c1cec11f44d4455f6c389b34a7788ccd7b05c6ab72ae8641e9 Jun 10 12:49:18.192007 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:18.191978 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb"] Jun 10 12:49:18.194829 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:49:18.194789 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0a2eb0f_40e5_4697_943e_299c4facfcba.slice/crio-4c784c039f2c94733ff4f0d19e556f0ae74eb41e91974302dc3e7c6ea6a9fada WatchSource:0}: Error finding container 4c784c039f2c94733ff4f0d19e556f0ae74eb41e91974302dc3e7c6ea6a9fada: Status 404 returned error can't find the container with id 4c784c039f2c94733ff4f0d19e556f0ae74eb41e91974302dc3e7c6ea6a9fada Jun 10 12:49:19.093036 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:19.092983 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" event={"ID":"f0a2eb0f-40e5-4697-943e-299c4facfcba","Type":"ContainerStarted","Data":"4c784c039f2c94733ff4f0d19e556f0ae74eb41e91974302dc3e7c6ea6a9fada"} Jun 10 12:49:19.096208 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:19.096178 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" event={"ID":"ff89facf-123c-4f6f-8fb8-d13eff6410b9","Type":"ContainerStarted","Data":"c624507447deb2c1cec11f44d4455f6c389b34a7788ccd7b05c6ab72ae8641e9"} Jun 10 12:49:24.113708 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:24.113671 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" event={"ID":"f0a2eb0f-40e5-4697-943e-299c4facfcba","Type":"ContainerStarted","Data":"b0a1e4505794d44bb1ef78d374d543abd202c1c627cdb170bc8b6ffb1fbace69"} Jun 10 12:49:24.114093 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:24.113719 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" event={"ID":"f0a2eb0f-40e5-4697-943e-299c4facfcba","Type":"ContainerStarted","Data":"6963260e4b1b831c24a9db904c15f57f6a3eea390c1d7f60586941bd827b30cc"} Jun 10 12:49:24.114093 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:24.113737 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" event={"ID":"f0a2eb0f-40e5-4697-943e-299c4facfcba","Type":"ContainerStarted","Data":"fe494f38e6ccb3d4d01fc27b9e61c2aed80ea8d566884cb26c98a554ff89c941"} Jun 10 12:49:24.115181 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:24.115157 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" event={"ID":"ff89facf-123c-4f6f-8fb8-d13eff6410b9","Type":"ContainerStarted","Data":"7bdda9878839e69b085acc7ede9fe431fb209f1a3f051e094287e4666296e666"} Jun 10 12:49:24.115375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:24.115359 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:24.116906 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:24.116889 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:49:24.137357 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:24.137299 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" podStartSLOduration=1.4041880930000001 podStartE2EDuration="7.137277141s" totalImagesPullingTime="5.733089048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:24.134181724 +0000 UTC m=+79.840485913" watchObservedRunningTime="2026-06-10 12:49:24.137277141 +0000 UTC m=+79.843581331" Jun 10 12:49:24.153509 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:24.153461 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" podStartSLOduration=1.385975199 podStartE2EDuration="7.153446655s" totalImagesPullingTime="5.767471456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:24.152841931 +0000 UTC m=+79.859146119" watchObservedRunningTime="2026-06-10 12:49:24.153446655 +0000 UTC m=+79.859750841" Jun 10 12:49:43.530785 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:43.530678 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:49:43.531197 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:49:43.530859 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:49:43.531197 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:49:43.530924 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:50:47.530908297 +0000 UTC m=+163.237212463 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:49:47.086511 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:49:47.086476 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v8wvp" Jun 10 12:50:14.544052 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:50:14.544015 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:50:14.544550 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:50:14.544150 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:50:14.544550 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:50:14.544206 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:52:16.544191557 +0000 UTC m=+252.250495725 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : secret "metrics-daemon-secret" not found Jun 10 12:50:42.688543 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:50:42.688494 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-v78tb" podUID="55dd8921-54a8-4d4f-beb9-ffd85f02ee5c" Jun 10 12:50:42.813174 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:50:42.813131 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:50:47.579257 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:50:47.579218 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:50:47.579708 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:50:47.579400 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:50:47.579708 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:50:47.579473 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls podName:55dd8921-54a8-4d4f-beb9-ffd85f02ee5c nodeName:}" failed. No retries permitted until 2026-06-10 12:52:49.579456382 +0000 UTC m=+285.285760549 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls") pod "dns-default-v78tb" (UID: "55dd8921-54a8-4d4f-beb9-ffd85f02ee5c") : secret "dns-default-metrics-tls" not found Jun 10 12:50:53.786219 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:50:53.786139 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:50:53.786658 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:50:53.786139 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v78tb" Jun 10 12:51:24.116589 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:51:24.116532 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" podUID="ff89facf-123c-4f6f-8fb8-d13eff6410b9" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.8:8000/readyz\": dial tcp 10.133.0.8:8000: connect: connection refused" Jun 10 12:51:24.464512 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:51:24.464479 2577 generic.go:356] "Generic (PLEG): container finished" podID="ff89facf-123c-4f6f-8fb8-d13eff6410b9" containerID="7bdda9878839e69b085acc7ede9fe431fb209f1a3f051e094287e4666296e666" exitCode=1 Jun 10 12:51:24.464679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:51:24.464532 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" event={"ID":"ff89facf-123c-4f6f-8fb8-d13eff6410b9","Type":"ContainerDied","Data":"7bdda9878839e69b085acc7ede9fe431fb209f1a3f051e094287e4666296e666"} Jun 10 12:51:24.464854 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:51:24.464841 2577 scope.go:117] "RemoveContainer" containerID="7bdda9878839e69b085acc7ede9fe431fb209f1a3f051e094287e4666296e666" Jun 10 12:51:25.468879 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:51:25.468847 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" event={"ID":"ff89facf-123c-4f6f-8fb8-d13eff6410b9","Type":"ContainerStarted","Data":"5dcd26b6b7cd1bad63c1c785313985359247f9a6a58df98812c6636d77cce3d3"} Jun 10 12:51:25.469292 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:51:25.469133 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:51:25.469663 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:51:25.469640 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f8bcd54-fkph7" Jun 10 12:52:16.606633 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:16.606594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:52:16.607124 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:52:16.606651 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:52:16.607124 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:52:16.606723 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs podName:1d88de4b-5158-4a97-b796-74a12f088e70 nodeName:}" failed. No retries permitted until 2026-06-10 12:54:18.60670825 +0000 UTC m=+374.313012419 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs") pod "network-metrics-daemon-gcbmf" (UID: "1d88de4b-5158-4a97-b796-74a12f088e70") : secret "metrics-daemon-secret" not found Jun 10 12:52:36.006130 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:36.006046 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wqrsz_1cd4f44a-0a1e-4710-a68c-be170c527c22/dns-node-resolver/0.log" Jun 10 12:52:36.807255 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:36.807225 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k7tz2_8136e2a8-4ddc-46fa-b368-c6803750c135/node-ca/0.log" Jun 10 12:52:37.807094 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:37.807066 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sbvn9_513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1/serve-healthcheck-canary/0.log" Jun 10 12:52:49.631191 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:49.631149 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:52:49.633430 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:49.633411 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55dd8921-54a8-4d4f-beb9-ffd85f02ee5c-metrics-tls\") pod \"dns-default-v78tb\" (UID: \"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c\") " pod="openshift-dns/dns-default-v78tb" Jun 10 12:52:49.889912 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:49.889835 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s86xl\"" Jun 10 12:52:49.897296 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:49.897277 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v78tb" Jun 10 12:52:50.021659 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:50.021636 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v78tb"] Jun 10 12:52:50.024049 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:52:50.024023 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55dd8921_54a8_4d4f_beb9_ffd85f02ee5c.slice/crio-95ef4781a86c52b16523b6393f309a2c0ec3136b0507606b6ff5c20bdef8afc1 WatchSource:0}: Error finding container 95ef4781a86c52b16523b6393f309a2c0ec3136b0507606b6ff5c20bdef8afc1: Status 404 returned error can't find the container with id 95ef4781a86c52b16523b6393f309a2c0ec3136b0507606b6ff5c20bdef8afc1 Jun 10 12:52:50.714371 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:50.714327 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v78tb" event={"ID":"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c","Type":"ContainerStarted","Data":"95ef4781a86c52b16523b6393f309a2c0ec3136b0507606b6ff5c20bdef8afc1"} Jun 10 12:52:52.723092 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:52.723055 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v78tb" event={"ID":"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c","Type":"ContainerStarted","Data":"ac39b2f6bac9623713d7fb4f99ae30a9ba70bfd9ea88ddd3158fb4ae9f4222b5"} Jun 10 12:52:52.723517 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:52.723109 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v78tb" event={"ID":"55dd8921-54a8-4d4f-beb9-ffd85f02ee5c","Type":"ContainerStarted","Data":"0c58e5505736cce253d10f6f57bfe10280b0aa8123b9331174935a1f85b46849"} Jun 10 12:52:52.723517 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:52.723182 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v78tb" Jun 10 12:52:52.741297 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:52.741250 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v78tb" podStartSLOduration=252.118339149 podStartE2EDuration="4m13.741236199s" totalImagesPullingTime="1.62289705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:52:52.740888674 +0000 UTC m=+288.447192863" watchObservedRunningTime="2026-06-10 12:52:52.741236199 +0000 UTC m=+288.447540388" Jun 10 12:52:55.150532 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.150496 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w2d49"] Jun 10 12:52:55.153447 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.153432 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.158087 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.158068 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 10 12:52:55.159377 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.159357 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 10 12:52:55.159498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.159384 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 10 12:52:55.159909 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.159769 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 10 12:52:55.159909 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.159856 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9c2ln\"" Jun 10 12:52:55.169796 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.169775 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w2d49"] Jun 10 12:52:55.174034 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.174003 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whwh7\" (UniqueName: \"kubernetes.io/projected/503b2798-f51f-4e92-9b18-eb2187fc6d7c-kube-api-access-whwh7\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.174034 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.174035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/503b2798-f51f-4e92-9b18-eb2187fc6d7c-data-volume\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.174160 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.174068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/503b2798-f51f-4e92-9b18-eb2187fc6d7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.174160 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.174114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/503b2798-f51f-4e92-9b18-eb2187fc6d7c-crio-socket\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.174160 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.174137 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/503b2798-f51f-4e92-9b18-eb2187fc6d7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.275435 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.275395 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/503b2798-f51f-4e92-9b18-eb2187fc6d7c-crio-socket\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.275435 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.275436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/503b2798-f51f-4e92-9b18-eb2187fc6d7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.275642 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.275521 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/503b2798-f51f-4e92-9b18-eb2187fc6d7c-crio-socket\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.275642 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.275560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whwh7\" (UniqueName: \"kubernetes.io/projected/503b2798-f51f-4e92-9b18-eb2187fc6d7c-kube-api-access-whwh7\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.275642 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.275595 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/503b2798-f51f-4e92-9b18-eb2187fc6d7c-data-volume\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.275642 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.275612 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/503b2798-f51f-4e92-9b18-eb2187fc6d7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.275969 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.275947 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/503b2798-f51f-4e92-9b18-eb2187fc6d7c-data-volume\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.276177 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.276160 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/503b2798-f51f-4e92-9b18-eb2187fc6d7c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.277773 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.277747 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/503b2798-f51f-4e92-9b18-eb2187fc6d7c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.286051 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.286028 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whwh7\" (UniqueName: \"kubernetes.io/projected/503b2798-f51f-4e92-9b18-eb2187fc6d7c-kube-api-access-whwh7\") pod \"insights-runtime-extractor-w2d49\" (UID: \"503b2798-f51f-4e92-9b18-eb2187fc6d7c\") " pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.462244 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.462156 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w2d49" Jun 10 12:52:55.614197 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.614168 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w2d49"] Jun 10 12:52:55.616975 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:52:55.616947 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503b2798_f51f_4e92_9b18_eb2187fc6d7c.slice/crio-fd9cbdc9a9e81043310832c84155ee8fa9623dea216b1add0eca7a4f0a1023fc WatchSource:0}: Error finding container fd9cbdc9a9e81043310832c84155ee8fa9623dea216b1add0eca7a4f0a1023fc: Status 404 returned error can't find the container with id fd9cbdc9a9e81043310832c84155ee8fa9623dea216b1add0eca7a4f0a1023fc Jun 10 12:52:55.733103 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.733023 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w2d49" event={"ID":"503b2798-f51f-4e92-9b18-eb2187fc6d7c","Type":"ContainerStarted","Data":"10ba0f87281d1e3e499dd29a16aead1d4a08d0d729bc22fbd1fc42a07d878504"} Jun 10 12:52:55.733103 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:55.733076 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w2d49" event={"ID":"503b2798-f51f-4e92-9b18-eb2187fc6d7c","Type":"ContainerStarted","Data":"fd9cbdc9a9e81043310832c84155ee8fa9623dea216b1add0eca7a4f0a1023fc"} Jun 10 12:52:56.738970 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:56.738889 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w2d49" event={"ID":"503b2798-f51f-4e92-9b18-eb2187fc6d7c","Type":"ContainerStarted","Data":"326e190517ee904826848f693557dc0eba711267acce89aa2277cb0b4832e4c2"} Jun 10 12:52:56.787387 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:52:56.787349 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-gcbmf" podUID="1d88de4b-5158-4a97-b796-74a12f088e70" Jun 10 12:52:58.749230 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:52:58.749197 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w2d49" event={"ID":"503b2798-f51f-4e92-9b18-eb2187fc6d7c","Type":"ContainerStarted","Data":"078c45280a26b680640dfca2cd0c23f479fc2b690b555b8403b987f6647f7947"} Jun 10 12:53:02.728338 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:02.728311 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v78tb" Jun 10 12:53:02.749604 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:02.749553 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w2d49" podStartSLOduration=5.637825392 podStartE2EDuration="7.74953906s" totalImagesPullingTime="2.111713668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:52:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:52:58.775363257 +0000 UTC m=+294.481667446" watchObservedRunningTime="2026-06-10 12:53:02.74953906 +0000 UTC m=+298.455843249" Jun 10 12:53:04.730525 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:04.730495 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:53:04.731615 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:04.731595 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:53:04.733863 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:04.733843 2577 kubelet.go:1648] "Image garbage collection succeeded" Jun 10 12:53:06.399685 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.399647 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-675c9d99fc-ptbsk"] Jun 10 12:53:06.402901 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.402885 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.407902 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.407875 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 10 12:53:06.407902 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.407890 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8glqp\"" Jun 10 12:53:06.408092 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.407877 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 10 12:53:06.408092 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.407877 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 10 12:53:06.408240 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.408221 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 10 12:53:06.408356 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.408251 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 10 12:53:06.408356 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.408263 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 10 12:53:06.408356 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.408342 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 10 12:53:06.414560 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.414542 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-675c9d99fc-ptbsk"] Jun 10 12:53:06.450107 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.450081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkh62\" (UniqueName: \"kubernetes.io/projected/a56840ab-0947-417e-aaf6-a405646778b7-kube-api-access-qkh62\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.450292 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.450117 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-oauth-serving-cert\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.450292 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.450150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-console-config\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.450292 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.450177 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-serving-cert\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.450292 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.450227 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-oauth-config\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.450292 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.450283 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-service-ca\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.551442 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.551409 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-service-ca\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.551597 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.551453 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkh62\" (UniqueName: \"kubernetes.io/projected/a56840ab-0947-417e-aaf6-a405646778b7-kube-api-access-qkh62\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.551597 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.551483 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-oauth-serving-cert\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.551597 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.551512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-console-config\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.551597 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.551538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-serving-cert\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.551597 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.551588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-oauth-config\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.552257 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.552230 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-service-ca\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.552359 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.552282 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-oauth-serving-cert\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.552359 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.552283 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-console-config\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.553983 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.553954 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-oauth-config\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.554089 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.554064 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-serving-cert\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.560547 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.560527 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkh62\" (UniqueName: \"kubernetes.io/projected/a56840ab-0947-417e-aaf6-a405646778b7-kube-api-access-qkh62\") pod \"console-675c9d99fc-ptbsk\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.712024 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.711929 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:06.850226 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.850192 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-675c9d99fc-ptbsk"] Jun 10 12:53:06.855889 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:53:06.855862 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56840ab_0947_417e_aaf6_a405646778b7.slice/crio-c3fc216c14f5092dee9a2b59eed97727707a2aaf962cabcc6db772caf55da2b3 WatchSource:0}: Error finding container c3fc216c14f5092dee9a2b59eed97727707a2aaf962cabcc6db772caf55da2b3: Status 404 returned error can't find the container with id c3fc216c14f5092dee9a2b59eed97727707a2aaf962cabcc6db772caf55da2b3 Jun 10 12:53:06.857956 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:06.857941 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:53:07.469703 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.469670 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xn9xr"] Jun 10 12:53:07.472957 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.472903 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.475450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.475412 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 10 12:53:07.475606 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.475478 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 10 12:53:07.475718 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.475704 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 10 12:53:07.476649 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.476627 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 10 12:53:07.476844 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.476673 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-29x2b\"" Jun 10 12:53:07.476844 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.476764 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 10 12:53:07.477292 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.477253 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 10 12:53:07.560777 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.560736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d116ca5f-e850-497f-a8ac-48f845f4cc1f-metrics-client-ca\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.560956 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.560796 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.560956 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.560912 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-tls\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.561048 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.560955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm2mf\" (UniqueName: \"kubernetes.io/projected/d116ca5f-e850-497f-a8ac-48f845f4cc1f-kube-api-access-nm2mf\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.561048 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.560996 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-root\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.561048 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.561025 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-sys\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.561202 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.561115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-textfile\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.561202 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.561151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.561202 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.561184 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-wtmp\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.661940 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.661903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-textfile\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662117 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.661948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662117 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.661982 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-wtmp\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662117 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d116ca5f-e850-497f-a8ac-48f845f4cc1f-metrics-client-ca\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662117 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662046 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662117 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662074 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-tls\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662117 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662112 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm2mf\" (UniqueName: \"kubernetes.io/projected/d116ca5f-e850-497f-a8ac-48f845f4cc1f-kube-api-access-nm2mf\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662149 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-wtmp\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-root\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662227 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-sys\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662292 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-root\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662333 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d116ca5f-e850-497f-a8ac-48f845f4cc1f-sys\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662354 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-textfile\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662786 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662660 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.662883 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.662827 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d116ca5f-e850-497f-a8ac-48f845f4cc1f-metrics-client-ca\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.664761 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.664733 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.665209 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.665188 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d116ca5f-e850-497f-a8ac-48f845f4cc1f-node-exporter-tls\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.671167 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.671144 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm2mf\" (UniqueName: \"kubernetes.io/projected/d116ca5f-e850-497f-a8ac-48f845f4cc1f-kube-api-access-nm2mf\") pod \"node-exporter-xn9xr\" (UID: \"d116ca5f-e850-497f-a8ac-48f845f4cc1f\") " pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.777189 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.777100 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675c9d99fc-ptbsk" event={"ID":"a56840ab-0947-417e-aaf6-a405646778b7","Type":"ContainerStarted","Data":"c3fc216c14f5092dee9a2b59eed97727707a2aaf962cabcc6db772caf55da2b3"} Jun 10 12:53:07.784377 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:07.784349 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xn9xr" Jun 10 12:53:07.796202 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:53:07.796164 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd116ca5f_e850_497f_a8ac_48f845f4cc1f.slice/crio-7bcfe3ca2e835c9b4fe50eac0c28783894b6b265dc64f66f2b7521dce33f906d WatchSource:0}: Error finding container 7bcfe3ca2e835c9b4fe50eac0c28783894b6b265dc64f66f2b7521dce33f906d: Status 404 returned error can't find the container with id 7bcfe3ca2e835c9b4fe50eac0c28783894b6b265dc64f66f2b7521dce33f906d Jun 10 12:53:08.051476 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:08.051385 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" podUID="f0a2eb0f-40e5-4697-943e-299c4facfcba" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 10 12:53:08.781721 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:08.781684 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xn9xr" event={"ID":"d116ca5f-e850-497f-a8ac-48f845f4cc1f","Type":"ContainerStarted","Data":"7bcfe3ca2e835c9b4fe50eac0c28783894b6b265dc64f66f2b7521dce33f906d"} Jun 10 12:53:09.786146 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:09.785830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:53:09.787203 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:09.787175 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675c9d99fc-ptbsk" event={"ID":"a56840ab-0947-417e-aaf6-a405646778b7","Type":"ContainerStarted","Data":"560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f"} Jun 10 12:53:09.788694 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:09.788667 2577 generic.go:356] "Generic (PLEG): container finished" podID="d116ca5f-e850-497f-a8ac-48f845f4cc1f" containerID="5a2a450bb92d9d2ffaff703092478ab734cd5ae8eebe3aa61376f45f668035d5" exitCode=0 Jun 10 12:53:09.788774 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:09.788718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xn9xr" event={"ID":"d116ca5f-e850-497f-a8ac-48f845f4cc1f","Type":"ContainerDied","Data":"5a2a450bb92d9d2ffaff703092478ab734cd5ae8eebe3aa61376f45f668035d5"} Jun 10 12:53:09.835626 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:09.835574 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-675c9d99fc-ptbsk" podStartSLOduration=1.048612013 podStartE2EDuration="3.835553821s" totalImagesPullingTime="2.786941808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:09.835339993 +0000 UTC m=+305.541644182" watchObservedRunningTime="2026-06-10 12:53:09.835553821 +0000 UTC m=+305.541858011" Jun 10 12:53:10.795870 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:10.795831 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xn9xr" event={"ID":"d116ca5f-e850-497f-a8ac-48f845f4cc1f","Type":"ContainerStarted","Data":"9a38378935865eb563be00e0bea10fe0fcd77b44398edd3e5bd8d0a0ca58de17"} Jun 10 12:53:10.796320 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:10.795884 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xn9xr" event={"ID":"d116ca5f-e850-497f-a8ac-48f845f4cc1f","Type":"ContainerStarted","Data":"6c2e7799600bc315ab27176106f66f53f2271880ae21f108bbf2d8cefd239857"} Jun 10 12:53:11.817364 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.817309 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xn9xr" podStartSLOduration=2.9098157369999997 podStartE2EDuration="4.817290832s" totalImagesPullingTime="1.840493622s" totalInitContainerRuntime="66.981473ms" isStatefulPod=true podCreationTimestamp="2026-06-10 12:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:10.818309324 +0000 UTC m=+306.524613514" watchObservedRunningTime="2026-06-10 12:53:11.817290832 +0000 UTC m=+307.523595020" Jun 10 12:53:11.817733 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.817625 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d98576f89-87kbh"] Jun 10 12:53:11.819398 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.819382 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:11.821709 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.821683 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 10 12:53:11.821905 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.821732 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 10 12:53:11.822142 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.822126 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-klst4\"" Jun 10 12:53:11.822391 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.822377 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 10 12:53:11.823541 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.823523 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b1092anj53mt8\"" Jun 10 12:53:11.823781 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.823768 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 10 12:53:11.832441 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.832420 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d98576f89-87kbh"] Jun 10 12:53:11.899438 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.899406 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-metrics-server-audit-profiles\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:11.899578 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.899452 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-secret-metrics-server-tls\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:11.899578 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.899469 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-audit-log\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:11.899578 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.899535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-secret-metrics-server-client-certs\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:11.899779 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.899760 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gnqh\" (UniqueName: \"kubernetes.io/projected/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-kube-api-access-8gnqh\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:11.899862 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.899838 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:11.899917 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:11.899868 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-client-ca-bundle\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.001114 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.001073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-secret-metrics-server-tls\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.001114 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.001118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-audit-log\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.001315 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.001284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-secret-metrics-server-client-certs\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.001407 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.001389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gnqh\" (UniqueName: \"kubernetes.io/projected/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-kube-api-access-8gnqh\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.001460 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.001445 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.001513 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.001474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-client-ca-bundle\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.001564 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.001507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-metrics-server-audit-profiles\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.001564 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.001525 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-audit-log\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.002213 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.002177 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.002443 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.002423 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-metrics-server-audit-profiles\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.003526 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.003507 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-secret-metrics-server-tls\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.003625 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.003596 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-secret-metrics-server-client-certs\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.003905 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.003886 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-client-ca-bundle\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.011161 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.011140 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gnqh\" (UniqueName: \"kubernetes.io/projected/c115d7e5-d1fd-4255-a140-35f67dc5f3d0-kube-api-access-8gnqh\") pod \"metrics-server-6d98576f89-87kbh\" (UID: \"c115d7e5-d1fd-4255-a140-35f67dc5f3d0\") " pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.128745 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.128713 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:12.256054 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.256021 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d98576f89-87kbh"] Jun 10 12:53:12.258531 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:53:12.258505 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc115d7e5_d1fd_4255_a140_35f67dc5f3d0.slice/crio-2aab9f81655d0000c15c4fd5c49ab746749bb873d6d739d0f6f29c7592cda01b WatchSource:0}: Error finding container 2aab9f81655d0000c15c4fd5c49ab746749bb873d6d739d0f6f29c7592cda01b: Status 404 returned error can't find the container with id 2aab9f81655d0000c15c4fd5c49ab746749bb873d6d739d0f6f29c7592cda01b Jun 10 12:53:12.805109 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:12.805076 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" event={"ID":"c115d7e5-d1fd-4255-a140-35f67dc5f3d0","Type":"ContainerStarted","Data":"2aab9f81655d0000c15c4fd5c49ab746749bb873d6d739d0f6f29c7592cda01b"} Jun 10 12:53:13.809791 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:13.809708 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" event={"ID":"c115d7e5-d1fd-4255-a140-35f67dc5f3d0","Type":"ContainerStarted","Data":"8c1238fdcfa49ce9463b9283c1e7ed0d1f25bdd943c33436be45541eb5df9ebf"} Jun 10 12:53:13.828748 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:13.828699 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" podStartSLOduration=1.626471971 podStartE2EDuration="2.828683798s" totalImagesPullingTime="1.202211827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:13.827924192 +0000 UTC m=+309.534228382" watchObservedRunningTime="2026-06-10 12:53:13.828683798 +0000 UTC m=+309.534987987" Jun 10 12:53:16.712986 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:16.712945 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:16.713376 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:16.713013 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:16.717824 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:16.717783 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:16.824372 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:16.824348 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:18.050891 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:18.050849 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" podUID="f0a2eb0f-40e5-4697-943e-299c4facfcba" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 10 12:53:20.530993 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.530958 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7679b65b44-wkdgx"] Jun 10 12:53:20.534043 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.534019 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.542425 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.542401 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 10 12:53:20.545847 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.545824 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7679b65b44-wkdgx"] Jun 10 12:53:20.674945 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.674909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-serving-cert\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.674945 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.674953 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-config\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.675180 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.674975 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7kng\" (UniqueName: \"kubernetes.io/projected/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-kube-api-access-p7kng\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.675180 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.675057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-oauth-config\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.675180 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.675098 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-oauth-serving-cert\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.675180 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.675137 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-service-ca\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.675180 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.675156 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-trusted-ca-bundle\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.775738 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.775701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-oauth-serving-cert\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.775738 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.775740 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-service-ca\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776005 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.775757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-trusted-ca-bundle\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776005 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.775798 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-serving-cert\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776005 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.775843 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-config\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776005 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.775870 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7kng\" (UniqueName: \"kubernetes.io/projected/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-kube-api-access-p7kng\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776005 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.775898 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-oauth-config\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776544 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.776506 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-oauth-serving-cert\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776664 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.776555 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-service-ca\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776664 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.776564 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-config\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.776826 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.776777 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-trusted-ca-bundle\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.778519 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.778490 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-oauth-config\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.778612 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.778593 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-serving-cert\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.786256 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.786203 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7kng\" (UniqueName: \"kubernetes.io/projected/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-kube-api-access-p7kng\") pod \"console-7679b65b44-wkdgx\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.843911 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.843872 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:20.971085 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:20.971051 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7679b65b44-wkdgx"] Jun 10 12:53:20.973831 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:53:20.973770 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf78b8b_e54f_4e9b_827e_e4d44300dcc2.slice/crio-17f0da5b7c4b04a034a223c04b27fa0ca80f8ca894c8cc9344e9c81a17db4cdd WatchSource:0}: Error finding container 17f0da5b7c4b04a034a223c04b27fa0ca80f8ca894c8cc9344e9c81a17db4cdd: Status 404 returned error can't find the container with id 17f0da5b7c4b04a034a223c04b27fa0ca80f8ca894c8cc9344e9c81a17db4cdd Jun 10 12:53:21.836572 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:21.836538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679b65b44-wkdgx" event={"ID":"caf78b8b-e54f-4e9b-827e-e4d44300dcc2","Type":"ContainerStarted","Data":"8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed"} Jun 10 12:53:21.836974 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:21.836597 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679b65b44-wkdgx" event={"ID":"caf78b8b-e54f-4e9b-827e-e4d44300dcc2","Type":"ContainerStarted","Data":"17f0da5b7c4b04a034a223c04b27fa0ca80f8ca894c8cc9344e9c81a17db4cdd"} Jun 10 12:53:21.856170 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:21.856116 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7679b65b44-wkdgx" podStartSLOduration=1.8560993890000002 podStartE2EDuration="1.856099389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:21.854784654 +0000 UTC m=+317.561088972" watchObservedRunningTime="2026-06-10 12:53:21.856099389 +0000 UTC m=+317.562403578" Jun 10 12:53:28.050824 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:28.050773 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" podUID="f0a2eb0f-40e5-4697-943e-299c4facfcba" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 10 12:53:28.051276 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:28.050858 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" Jun 10 12:53:28.051336 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:28.051307 2577 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"b0a1e4505794d44bb1ef78d374d543abd202c1c627cdb170bc8b6ffb1fbace69"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 10 12:53:28.051372 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:28.051355 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" podUID="f0a2eb0f-40e5-4697-943e-299c4facfcba" containerName="service-proxy" containerID="cri-o://b0a1e4505794d44bb1ef78d374d543abd202c1c627cdb170bc8b6ffb1fbace69" gracePeriod=30 Jun 10 12:53:28.860961 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:28.860925 2577 generic.go:356] "Generic (PLEG): container finished" podID="f0a2eb0f-40e5-4697-943e-299c4facfcba" containerID="b0a1e4505794d44bb1ef78d374d543abd202c1c627cdb170bc8b6ffb1fbace69" exitCode=2 Jun 10 12:53:28.861126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:28.860998 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" event={"ID":"f0a2eb0f-40e5-4697-943e-299c4facfcba","Type":"ContainerDied","Data":"b0a1e4505794d44bb1ef78d374d543abd202c1c627cdb170bc8b6ffb1fbace69"} Jun 10 12:53:28.861126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:28.861042 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58b7b76b55-m4gxb" event={"ID":"f0a2eb0f-40e5-4697-943e-299c4facfcba","Type":"ContainerStarted","Data":"d467ebff0db3fee27fc9ce739733696366a5232ad40ace1c8d09cbdc762f7b74"} Jun 10 12:53:30.844317 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:30.844282 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:30.844692 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:30.844343 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:30.849180 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:30.849155 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:30.872158 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:30.872134 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:53:30.958155 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:30.958122 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-675c9d99fc-ptbsk"] Jun 10 12:53:32.129215 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:32.129164 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:32.129638 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:32.129491 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:52.134385 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:52.134296 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:52.138160 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:52.138133 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d98576f89-87kbh" Jun 10 12:53:55.981857 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:55.981737 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-675c9d99fc-ptbsk" podUID="a56840ab-0947-417e-aaf6-a405646778b7" containerName="console" containerID="cri-o://560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f" gracePeriod=15 Jun 10 12:53:56.217297 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.217273 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:56.245931 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.245865 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkh62\" (UniqueName: \"kubernetes.io/projected/a56840ab-0947-417e-aaf6-a405646778b7-kube-api-access-qkh62\") pod \"a56840ab-0947-417e-aaf6-a405646778b7\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " Jun 10 12:53:56.246083 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.245939 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-service-ca\") pod \"a56840ab-0947-417e-aaf6-a405646778b7\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " Jun 10 12:53:56.246083 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.245959 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-console-config\") pod \"a56840ab-0947-417e-aaf6-a405646778b7\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " Jun 10 12:53:56.246083 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.245981 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-serving-cert\") pod \"a56840ab-0947-417e-aaf6-a405646778b7\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " Jun 10 12:53:56.246083 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.246000 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-oauth-serving-cert\") pod \"a56840ab-0947-417e-aaf6-a405646778b7\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " Jun 10 12:53:56.246083 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.246015 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-oauth-config\") pod \"a56840ab-0947-417e-aaf6-a405646778b7\" (UID: \"a56840ab-0947-417e-aaf6-a405646778b7\") " Jun 10 12:53:56.246429 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.246386 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-console-config" (OuterVolumeSpecName: "console-config") pod "a56840ab-0947-417e-aaf6-a405646778b7" (UID: "a56840ab-0947-417e-aaf6-a405646778b7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:53:56.246429 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.246411 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a56840ab-0947-417e-aaf6-a405646778b7" (UID: "a56840ab-0947-417e-aaf6-a405646778b7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:53:56.246615 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.246423 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-service-ca" (OuterVolumeSpecName: "service-ca") pod "a56840ab-0947-417e-aaf6-a405646778b7" (UID: "a56840ab-0947-417e-aaf6-a405646778b7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:53:56.248340 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.248312 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a56840ab-0947-417e-aaf6-a405646778b7" (UID: "a56840ab-0947-417e-aaf6-a405646778b7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:53:56.248439 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.248398 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a56840ab-0947-417e-aaf6-a405646778b7" (UID: "a56840ab-0947-417e-aaf6-a405646778b7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:53:56.248439 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.248407 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a56840ab-0947-417e-aaf6-a405646778b7-kube-api-access-qkh62" (OuterVolumeSpecName: "kube-api-access-qkh62") pod "a56840ab-0947-417e-aaf6-a405646778b7" (UID: "a56840ab-0947-417e-aaf6-a405646778b7"). InnerVolumeSpecName "kube-api-access-qkh62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:53:56.347125 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.347086 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkh62\" (UniqueName: \"kubernetes.io/projected/a56840ab-0947-417e-aaf6-a405646778b7-kube-api-access-qkh62\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:53:56.347125 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.347116 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-service-ca\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:53:56.347125 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.347128 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-console-config\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:53:56.347354 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.347139 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-serving-cert\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:53:56.347354 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.347152 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a56840ab-0947-417e-aaf6-a405646778b7-oauth-serving-cert\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:53:56.347354 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.347163 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a56840ab-0947-417e-aaf6-a405646778b7-console-oauth-config\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:53:56.954181 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.954142 2577 generic.go:356] "Generic (PLEG): container finished" podID="a56840ab-0947-417e-aaf6-a405646778b7" containerID="560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f" exitCode=0 Jun 10 12:53:56.954341 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.954195 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675c9d99fc-ptbsk" Jun 10 12:53:56.954341 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.954229 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675c9d99fc-ptbsk" event={"ID":"a56840ab-0947-417e-aaf6-a405646778b7","Type":"ContainerDied","Data":"560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f"} Jun 10 12:53:56.954341 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.954279 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675c9d99fc-ptbsk" event={"ID":"a56840ab-0947-417e-aaf6-a405646778b7","Type":"ContainerDied","Data":"c3fc216c14f5092dee9a2b59eed97727707a2aaf962cabcc6db772caf55da2b3"} Jun 10 12:53:56.954341 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.954300 2577 scope.go:117] "RemoveContainer" containerID="560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f" Jun 10 12:53:56.962094 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.962076 2577 scope.go:117] "RemoveContainer" containerID="560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f" Jun 10 12:53:56.962369 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:53:56.962342 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f\": container with ID starting with 560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f not found: ID does not exist" containerID="560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f" Jun 10 12:53:56.962431 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.962381 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f"} err="failed to get container status \"560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f\": rpc error: code = NotFound desc = could not find container \"560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f\": container with ID starting with 560e5790430113f171ad8524e40b6a1c6398dce219b0d5ba5790e3d38192e60f not found: ID does not exist" Jun 10 12:53:56.970878 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.970855 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-675c9d99fc-ptbsk"] Jun 10 12:53:56.973179 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:56.973159 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-675c9d99fc-ptbsk"] Jun 10 12:53:58.790197 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:53:58.790158 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a56840ab-0947-417e-aaf6-a405646778b7" path="/var/lib/kubelet/pods/a56840ab-0947-417e-aaf6-a405646778b7/volumes" Jun 10 12:54:18.637362 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:18.637324 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:54:18.639780 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:18.639747 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d88de4b-5158-4a97-b796-74a12f088e70-metrics-certs\") pod \"network-metrics-daemon-gcbmf\" (UID: \"1d88de4b-5158-4a97-b796-74a12f088e70\") " pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:54:18.789497 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:18.789469 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7d5dq\"" Jun 10 12:54:18.796929 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:18.796908 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcbmf" Jun 10 12:54:18.928976 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:18.928894 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gcbmf"] Jun 10 12:54:18.931818 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:54:18.931780 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d88de4b_5158_4a97_b796_74a12f088e70.slice/crio-d5d7125642a745ab29b88ff35f0bdd4d66238eb9a2e19c5253c186ac8d283bb6 WatchSource:0}: Error finding container d5d7125642a745ab29b88ff35f0bdd4d66238eb9a2e19c5253c186ac8d283bb6: Status 404 returned error can't find the container with id d5d7125642a745ab29b88ff35f0bdd4d66238eb9a2e19c5253c186ac8d283bb6 Jun 10 12:54:19.027542 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:19.027509 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gcbmf" event={"ID":"1d88de4b-5158-4a97-b796-74a12f088e70","Type":"ContainerStarted","Data":"d5d7125642a745ab29b88ff35f0bdd4d66238eb9a2e19c5253c186ac8d283bb6"} Jun 10 12:54:20.033515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:20.033476 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gcbmf" event={"ID":"1d88de4b-5158-4a97-b796-74a12f088e70","Type":"ContainerStarted","Data":"ac6b6a258abf3842923e1ea50e03f4fe5b7520803b57469d2e1ddef5d069daaa"} Jun 10 12:54:20.033954 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:20.033534 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gcbmf" event={"ID":"1d88de4b-5158-4a97-b796-74a12f088e70","Type":"ContainerStarted","Data":"b10a1a52248c22cb6c66c3ef4c6344560f90626851d9ecb8898c31edc9b69477"} Jun 10 12:54:33.346163 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.346106 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gcbmf" podStartSLOduration=388.490885707 podStartE2EDuration="6m29.346087599s" totalImagesPullingTime="855.201892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:20.059571335 +0000 UTC m=+375.765875524" watchObservedRunningTime="2026-06-10 12:54:33.346087599 +0000 UTC m=+389.052391815" Jun 10 12:54:33.346688 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.346670 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86cfc994bd-mjdxc"] Jun 10 12:54:33.346940 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.346928 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a56840ab-0947-417e-aaf6-a405646778b7" containerName="console" Jun 10 12:54:33.346986 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.346941 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a56840ab-0947-417e-aaf6-a405646778b7" containerName="console" Jun 10 12:54:33.347027 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.346987 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a56840ab-0947-417e-aaf6-a405646778b7" containerName="console" Jun 10 12:54:33.353255 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.353230 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.364178 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.364154 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86cfc994bd-mjdxc"] Jun 10 12:54:33.443056 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.443017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-serving-cert\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.443056 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.443059 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-service-ca\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.443268 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.443084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-oauth-config\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.443268 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.443105 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxcls\" (UniqueName: \"kubernetes.io/projected/6d6f444f-4a19-4308-8858-5552aae7ea15-kube-api-access-fxcls\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.443268 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.443163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-trusted-ca-bundle\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.443268 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.443213 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-console-config\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.443401 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.443270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-oauth-serving-cert\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.544339 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.544300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-service-ca\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.544339 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.544337 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-oauth-config\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.544586 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.544353 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxcls\" (UniqueName: \"kubernetes.io/projected/6d6f444f-4a19-4308-8858-5552aae7ea15-kube-api-access-fxcls\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.544586 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.544493 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-trusted-ca-bundle\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.544586 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.544558 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-console-config\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.544740 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.544603 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-oauth-serving-cert\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.544740 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.544663 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-serving-cert\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.545156 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.545118 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-service-ca\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.545294 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.545277 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-console-config\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.545356 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.545305 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-oauth-serving-cert\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.545394 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.545367 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-trusted-ca-bundle\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.547475 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.547452 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-serving-cert\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.547566 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.547480 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-oauth-config\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.552734 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.552714 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxcls\" (UniqueName: \"kubernetes.io/projected/6d6f444f-4a19-4308-8858-5552aae7ea15-kube-api-access-fxcls\") pod \"console-86cfc994bd-mjdxc\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.661963 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.661927 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:33.795514 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:33.795479 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86cfc994bd-mjdxc"] Jun 10 12:54:33.798247 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:54:33.798218 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d6f444f_4a19_4308_8858_5552aae7ea15.slice/crio-b6d8ccebcb5f600a115c9a9ad9b47588de948de223c3ac517d574a955da5125b WatchSource:0}: Error finding container b6d8ccebcb5f600a115c9a9ad9b47588de948de223c3ac517d574a955da5125b: Status 404 returned error can't find the container with id b6d8ccebcb5f600a115c9a9ad9b47588de948de223c3ac517d574a955da5125b Jun 10 12:54:34.084613 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:34.084522 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cfc994bd-mjdxc" event={"ID":"6d6f444f-4a19-4308-8858-5552aae7ea15","Type":"ContainerStarted","Data":"a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b"} Jun 10 12:54:34.084613 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:34.084575 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cfc994bd-mjdxc" event={"ID":"6d6f444f-4a19-4308-8858-5552aae7ea15","Type":"ContainerStarted","Data":"b6d8ccebcb5f600a115c9a9ad9b47588de948de223c3ac517d574a955da5125b"} Jun 10 12:54:34.106203 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:34.106149 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86cfc994bd-mjdxc" podStartSLOduration=1.1061329500000001 podStartE2EDuration="1.10613295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:34.104975493 +0000 UTC m=+389.811279682" watchObservedRunningTime="2026-06-10 12:54:34.10613295 +0000 UTC m=+389.812437139" Jun 10 12:54:43.662865 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:43.662825 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:43.663456 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:43.662887 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:43.667638 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:43.667615 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:44.121217 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:44.121189 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 12:54:44.175140 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:54:44.175111 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7679b65b44-wkdgx"] Jun 10 12:55:09.200201 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.200159 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7679b65b44-wkdgx" podUID="caf78b8b-e54f-4e9b-827e-e4d44300dcc2" containerName="console" containerID="cri-o://8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed" gracePeriod=15 Jun 10 12:55:09.434688 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.434658 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:55:09.511428 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511343 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-trusted-ca-bundle\") pod \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " Jun 10 12:55:09.511428 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511387 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-serving-cert\") pod \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " Jun 10 12:55:09.511428 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511408 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7kng\" (UniqueName: \"kubernetes.io/projected/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-kube-api-access-p7kng\") pod \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " Jun 10 12:55:09.511428 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511430 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-oauth-config\") pod \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " Jun 10 12:55:09.511753 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511564 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-service-ca\") pod \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " Jun 10 12:55:09.511753 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511635 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-config\") pod \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " Jun 10 12:55:09.511753 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511663 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-oauth-serving-cert\") pod \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\" (UID: \"caf78b8b-e54f-4e9b-827e-e4d44300dcc2\") " Jun 10 12:55:09.511945 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511917 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "caf78b8b-e54f-4e9b-827e-e4d44300dcc2" (UID: "caf78b8b-e54f-4e9b-827e-e4d44300dcc2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:09.512005 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.511937 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-service-ca" (OuterVolumeSpecName: "service-ca") pod "caf78b8b-e54f-4e9b-827e-e4d44300dcc2" (UID: "caf78b8b-e54f-4e9b-827e-e4d44300dcc2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:09.512060 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.512012 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-config" (OuterVolumeSpecName: "console-config") pod "caf78b8b-e54f-4e9b-827e-e4d44300dcc2" (UID: "caf78b8b-e54f-4e9b-827e-e4d44300dcc2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:09.512267 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.512240 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "caf78b8b-e54f-4e9b-827e-e4d44300dcc2" (UID: "caf78b8b-e54f-4e9b-827e-e4d44300dcc2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:09.513614 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.513587 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "caf78b8b-e54f-4e9b-827e-e4d44300dcc2" (UID: "caf78b8b-e54f-4e9b-827e-e4d44300dcc2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:09.513704 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.513640 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-kube-api-access-p7kng" (OuterVolumeSpecName: "kube-api-access-p7kng") pod "caf78b8b-e54f-4e9b-827e-e4d44300dcc2" (UID: "caf78b8b-e54f-4e9b-827e-e4d44300dcc2"). InnerVolumeSpecName "kube-api-access-p7kng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:55:09.513704 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.513671 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "caf78b8b-e54f-4e9b-827e-e4d44300dcc2" (UID: "caf78b8b-e54f-4e9b-827e-e4d44300dcc2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:09.612942 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.612892 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-oauth-config\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:09.612942 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.612933 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-service-ca\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:09.612942 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.612945 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-config\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:09.612942 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.612954 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-oauth-serving-cert\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:09.612942 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.612963 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-trusted-ca-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:09.613227 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.612972 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-console-serving-cert\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:09.613227 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:09.612980 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7kng\" (UniqueName: \"kubernetes.io/projected/caf78b8b-e54f-4e9b-827e-e4d44300dcc2-kube-api-access-p7kng\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:10.217598 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.217562 2577 generic.go:356] "Generic (PLEG): container finished" podID="caf78b8b-e54f-4e9b-827e-e4d44300dcc2" containerID="8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed" exitCode=0 Jun 10 12:55:10.218004 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.217610 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679b65b44-wkdgx" Jun 10 12:55:10.218004 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.217611 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679b65b44-wkdgx" event={"ID":"caf78b8b-e54f-4e9b-827e-e4d44300dcc2","Type":"ContainerDied","Data":"8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed"} Jun 10 12:55:10.218004 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.217734 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679b65b44-wkdgx" event={"ID":"caf78b8b-e54f-4e9b-827e-e4d44300dcc2","Type":"ContainerDied","Data":"17f0da5b7c4b04a034a223c04b27fa0ca80f8ca894c8cc9344e9c81a17db4cdd"} Jun 10 12:55:10.218004 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.217760 2577 scope.go:117] "RemoveContainer" containerID="8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed" Jun 10 12:55:10.226017 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.225999 2577 scope.go:117] "RemoveContainer" containerID="8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed" Jun 10 12:55:10.226303 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:55:10.226279 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed\": container with ID starting with 8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed not found: ID does not exist" containerID="8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed" Jun 10 12:55:10.226385 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.226311 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed"} err="failed to get container status \"8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed\": rpc error: code = NotFound desc = could not find container \"8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed\": container with ID starting with 8df7b9b10b049a0685ad7972af6f20b0549a080ba8a5f9aef39f16b3536832ed not found: ID does not exist" Jun 10 12:55:10.240673 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.240647 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7679b65b44-wkdgx"] Jun 10 12:55:10.244908 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.244881 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7679b65b44-wkdgx"] Jun 10 12:55:10.791002 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:10.790968 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caf78b8b-e54f-4e9b-827e-e4d44300dcc2" path="/var/lib/kubelet/pods/caf78b8b-e54f-4e9b-827e-e4d44300dcc2/volumes" Jun 10 12:55:23.955835 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.955735 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld"] Jun 10 12:55:23.956261 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.955988 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf78b8b-e54f-4e9b-827e-e4d44300dcc2" containerName="console" Jun 10 12:55:23.956261 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.955999 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf78b8b-e54f-4e9b-827e-e4d44300dcc2" containerName="console" Jun 10 12:55:23.956261 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.956037 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf78b8b-e54f-4e9b-827e-e4d44300dcc2" containerName="console" Jun 10 12:55:23.958918 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.958902 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:23.961592 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.961570 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:55:23.962610 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.962591 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jcl4f\"" Jun 10 12:55:23.962687 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.962591 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:55:23.971753 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:23.971731 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld"] Jun 10 12:55:24.011094 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.011065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.011267 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.011130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.011267 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.011175 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rhh2\" (UniqueName: \"kubernetes.io/projected/05586829-987a-4662-82fe-0d2b3be24ca5-kube-api-access-9rhh2\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.112519 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.112483 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.112519 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.112522 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rhh2\" (UniqueName: \"kubernetes.io/projected/05586829-987a-4662-82fe-0d2b3be24ca5-kube-api-access-9rhh2\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.112698 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.112547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.112895 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.112880 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.112935 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.112921 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.121594 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.121566 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rhh2\" (UniqueName: \"kubernetes.io/projected/05586829-987a-4662-82fe-0d2b3be24ca5-kube-api-access-9rhh2\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.267602 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.267520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:24.396631 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:24.396593 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld"] Jun 10 12:55:24.399794 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:55:24.399766 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05586829_987a_4662_82fe_0d2b3be24ca5.slice/crio-ee857e455a0fe58eb93d188a835c6df16f4c19acbd95caf09a29c8936904b91b WatchSource:0}: Error finding container ee857e455a0fe58eb93d188a835c6df16f4c19acbd95caf09a29c8936904b91b: Status 404 returned error can't find the container with id ee857e455a0fe58eb93d188a835c6df16f4c19acbd95caf09a29c8936904b91b Jun 10 12:55:25.275586 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:25.275542 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" event={"ID":"05586829-987a-4662-82fe-0d2b3be24ca5","Type":"ContainerStarted","Data":"ee857e455a0fe58eb93d188a835c6df16f4c19acbd95caf09a29c8936904b91b"} Jun 10 12:55:34.309342 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:34.309297 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" event={"ID":"05586829-987a-4662-82fe-0d2b3be24ca5","Type":"ContainerStarted","Data":"a4c761f690cbe221d1f03d2236a14a8af083da727ca6fed98e39a93325bfa43d"} Jun 10 12:55:35.314536 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:35.314500 2577 generic.go:356] "Generic (PLEG): container finished" podID="05586829-987a-4662-82fe-0d2b3be24ca5" containerID="a4c761f690cbe221d1f03d2236a14a8af083da727ca6fed98e39a93325bfa43d" exitCode=0 Jun 10 12:55:35.314985 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:35.314540 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" event={"ID":"05586829-987a-4662-82fe-0d2b3be24ca5","Type":"ContainerDied","Data":"a4c761f690cbe221d1f03d2236a14a8af083da727ca6fed98e39a93325bfa43d"} Jun 10 12:55:37.323192 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:37.323153 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" event={"ID":"05586829-987a-4662-82fe-0d2b3be24ca5","Type":"ContainerStarted","Data":"2d92b8b826d577d087e5691428a6347fd41558907ac32bd9ecdd1855a92e8148"} Jun 10 12:55:38.328922 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:38.328888 2577 generic.go:356] "Generic (PLEG): container finished" podID="05586829-987a-4662-82fe-0d2b3be24ca5" containerID="2d92b8b826d577d087e5691428a6347fd41558907ac32bd9ecdd1855a92e8148" exitCode=0 Jun 10 12:55:38.329337 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:38.328934 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" event={"ID":"05586829-987a-4662-82fe-0d2b3be24ca5","Type":"ContainerDied","Data":"2d92b8b826d577d087e5691428a6347fd41558907ac32bd9ecdd1855a92e8148"} Jun 10 12:55:47.363302 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:47.363266 2577 generic.go:356] "Generic (PLEG): container finished" podID="05586829-987a-4662-82fe-0d2b3be24ca5" containerID="9052714a006c9cf3e3c340382b89bae282639d8e4ae6d57fc752f1c5b83ef984" exitCode=0 Jun 10 12:55:47.363666 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:47.363341 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" event={"ID":"05586829-987a-4662-82fe-0d2b3be24ca5","Type":"ContainerDied","Data":"9052714a006c9cf3e3c340382b89bae282639d8e4ae6d57fc752f1c5b83ef984"} Jun 10 12:55:48.484972 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.484945 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:48.620450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.620352 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rhh2\" (UniqueName: \"kubernetes.io/projected/05586829-987a-4662-82fe-0d2b3be24ca5-kube-api-access-9rhh2\") pod \"05586829-987a-4662-82fe-0d2b3be24ca5\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " Jun 10 12:55:48.620450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.620401 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-util\") pod \"05586829-987a-4662-82fe-0d2b3be24ca5\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " Jun 10 12:55:48.620450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.620434 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-bundle\") pod \"05586829-987a-4662-82fe-0d2b3be24ca5\" (UID: \"05586829-987a-4662-82fe-0d2b3be24ca5\") " Jun 10 12:55:48.621262 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.621228 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-bundle" (OuterVolumeSpecName: "bundle") pod "05586829-987a-4662-82fe-0d2b3be24ca5" (UID: "05586829-987a-4662-82fe-0d2b3be24ca5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:55:48.622619 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.622588 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05586829-987a-4662-82fe-0d2b3be24ca5-kube-api-access-9rhh2" (OuterVolumeSpecName: "kube-api-access-9rhh2") pod "05586829-987a-4662-82fe-0d2b3be24ca5" (UID: "05586829-987a-4662-82fe-0d2b3be24ca5"). InnerVolumeSpecName "kube-api-access-9rhh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:55:48.624788 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.624764 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-util" (OuterVolumeSpecName: "util") pod "05586829-987a-4662-82fe-0d2b3be24ca5" (UID: "05586829-987a-4662-82fe-0d2b3be24ca5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:55:48.721096 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.721058 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:48.721096 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.721087 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rhh2\" (UniqueName: \"kubernetes.io/projected/05586829-987a-4662-82fe-0d2b3be24ca5-kube-api-access-9rhh2\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:48.721096 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:48.721099 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05586829-987a-4662-82fe-0d2b3be24ca5-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:55:49.372789 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:49.372755 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" event={"ID":"05586829-987a-4662-82fe-0d2b3be24ca5","Type":"ContainerDied","Data":"ee857e455a0fe58eb93d188a835c6df16f4c19acbd95caf09a29c8936904b91b"} Jun 10 12:55:49.372789 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:49.372790 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgrmld" Jun 10 12:55:49.373007 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:55:49.372847 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee857e455a0fe58eb93d188a835c6df16f4c19acbd95caf09a29c8936904b91b" Jun 10 12:56:02.464815 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.464765 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w"] Jun 10 12:56:02.465220 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.465025 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05586829-987a-4662-82fe-0d2b3be24ca5" containerName="pull" Jun 10 12:56:02.465220 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.465035 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="05586829-987a-4662-82fe-0d2b3be24ca5" containerName="pull" Jun 10 12:56:02.465220 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.465045 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05586829-987a-4662-82fe-0d2b3be24ca5" containerName="util" Jun 10 12:56:02.465220 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.465050 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="05586829-987a-4662-82fe-0d2b3be24ca5" containerName="util" Jun 10 12:56:02.465220 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.465059 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05586829-987a-4662-82fe-0d2b3be24ca5" containerName="extract" Jun 10 12:56:02.465220 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.465064 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="05586829-987a-4662-82fe-0d2b3be24ca5" containerName="extract" Jun 10 12:56:02.465220 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.465101 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="05586829-987a-4662-82fe-0d2b3be24ca5" containerName="extract" Jun 10 12:56:02.471268 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.471250 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.474935 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.474906 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jcl4f\"" Jun 10 12:56:02.475489 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.475467 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:56:02.477014 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.476994 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:56:02.488712 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.488689 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w"] Jun 10 12:56:02.623432 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.623390 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsj8m\" (UniqueName: \"kubernetes.io/projected/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-kube-api-access-nsj8m\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.623610 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.623454 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.623610 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.623484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.724680 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.724579 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsj8m\" (UniqueName: \"kubernetes.io/projected/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-kube-api-access-nsj8m\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.724680 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.724662 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.724680 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.724682 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.725029 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.725013 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.725083 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.725063 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.733884 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.733862 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsj8m\" (UniqueName: \"kubernetes.io/projected/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-kube-api-access-nsj8m\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.779731 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.779702 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:02.907735 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:02.907704 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w"] Jun 10 12:56:02.911139 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:56:02.911104 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba1a57a_dd5b_4741_9f66_4ad316f2b544.slice/crio-0ffc576a818e15cd354c70b5df852e70a46698ea0ed9e46f19bd71979533dd35 WatchSource:0}: Error finding container 0ffc576a818e15cd354c70b5df852e70a46698ea0ed9e46f19bd71979533dd35: Status 404 returned error can't find the container with id 0ffc576a818e15cd354c70b5df852e70a46698ea0ed9e46f19bd71979533dd35 Jun 10 12:56:03.427562 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:03.427531 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerID="7872e8df251c509324e3093a0b390708a53e579b4b19a1925fc8eaf161364206" exitCode=0 Jun 10 12:56:03.427745 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:03.427596 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" event={"ID":"3ba1a57a-dd5b-4741-9f66-4ad316f2b544","Type":"ContainerDied","Data":"7872e8df251c509324e3093a0b390708a53e579b4b19a1925fc8eaf161364206"} Jun 10 12:56:03.427745 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:03.427637 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" event={"ID":"3ba1a57a-dd5b-4741-9f66-4ad316f2b544","Type":"ContainerStarted","Data":"0ffc576a818e15cd354c70b5df852e70a46698ea0ed9e46f19bd71979533dd35"} Jun 10 12:56:06.440546 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:06.440513 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerID="3acf1fd03ae5e6afaae3e3118b7b796b521e7bc0908e3ea7ae1c594114a22057" exitCode=0 Jun 10 12:56:06.440956 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:06.440567 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" event={"ID":"3ba1a57a-dd5b-4741-9f66-4ad316f2b544","Type":"ContainerDied","Data":"3acf1fd03ae5e6afaae3e3118b7b796b521e7bc0908e3ea7ae1c594114a22057"} Jun 10 12:56:07.073564 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.073526 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-2czqn"] Jun 10 12:56:07.076455 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.076434 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" Jun 10 12:56:07.082602 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.082553 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 10 12:56:07.082733 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.082673 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 10 12:56:07.083878 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.083857 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hl7h6\"" Jun 10 12:56:07.088939 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.088914 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-2czqn"] Jun 10 12:56:07.155330 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.155296 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfrl9\" (UniqueName: \"kubernetes.io/projected/d8afad34-7253-4f96-8b7b-2d575afbe902-kube-api-access-zfrl9\") pod \"cert-manager-cainjector-5d99449ffd-2czqn\" (UID: \"d8afad34-7253-4f96-8b7b-2d575afbe902\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" Jun 10 12:56:07.155502 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.155365 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8afad34-7253-4f96-8b7b-2d575afbe902-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-2czqn\" (UID: \"d8afad34-7253-4f96-8b7b-2d575afbe902\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" Jun 10 12:56:07.256638 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.256605 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8afad34-7253-4f96-8b7b-2d575afbe902-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-2czqn\" (UID: \"d8afad34-7253-4f96-8b7b-2d575afbe902\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" Jun 10 12:56:07.256784 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.256645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfrl9\" (UniqueName: \"kubernetes.io/projected/d8afad34-7253-4f96-8b7b-2d575afbe902-kube-api-access-zfrl9\") pod \"cert-manager-cainjector-5d99449ffd-2czqn\" (UID: \"d8afad34-7253-4f96-8b7b-2d575afbe902\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" Jun 10 12:56:07.265964 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.265933 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8afad34-7253-4f96-8b7b-2d575afbe902-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-2czqn\" (UID: \"d8afad34-7253-4f96-8b7b-2d575afbe902\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" Jun 10 12:56:07.266078 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.265992 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfrl9\" (UniqueName: \"kubernetes.io/projected/d8afad34-7253-4f96-8b7b-2d575afbe902-kube-api-access-zfrl9\") pod \"cert-manager-cainjector-5d99449ffd-2czqn\" (UID: \"d8afad34-7253-4f96-8b7b-2d575afbe902\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" Jun 10 12:56:07.385579 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.385547 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" Jun 10 12:56:07.447104 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.447069 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerID="51a072cbd60e46dcf72d2a71f2a32016dfdd07ddbb0412cda5d4f7415b4b5675" exitCode=0 Jun 10 12:56:07.447467 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.447141 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" event={"ID":"3ba1a57a-dd5b-4741-9f66-4ad316f2b544","Type":"ContainerDied","Data":"51a072cbd60e46dcf72d2a71f2a32016dfdd07ddbb0412cda5d4f7415b4b5675"} Jun 10 12:56:07.519418 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:07.519381 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-2czqn"] Jun 10 12:56:07.521991 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:56:07.521957 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8afad34_7253_4f96_8b7b_2d575afbe902.slice/crio-327798a590adc74084f69bd8de609fdf75f9ab3686bd24a8a9034fe0460d4447 WatchSource:0}: Error finding container 327798a590adc74084f69bd8de609fdf75f9ab3686bd24a8a9034fe0460d4447: Status 404 returned error can't find the container with id 327798a590adc74084f69bd8de609fdf75f9ab3686bd24a8a9034fe0460d4447 Jun 10 12:56:08.452765 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.452726 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" event={"ID":"d8afad34-7253-4f96-8b7b-2d575afbe902","Type":"ContainerStarted","Data":"327798a590adc74084f69bd8de609fdf75f9ab3686bd24a8a9034fe0460d4447"} Jun 10 12:56:08.595338 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.595310 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:08.667434 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.667401 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-util\") pod \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " Jun 10 12:56:08.667598 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.667448 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsj8m\" (UniqueName: \"kubernetes.io/projected/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-kube-api-access-nsj8m\") pod \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " Jun 10 12:56:08.667598 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.667492 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-bundle\") pod \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\" (UID: \"3ba1a57a-dd5b-4741-9f66-4ad316f2b544\") " Jun 10 12:56:08.668037 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.668002 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-bundle" (OuterVolumeSpecName: "bundle") pod "3ba1a57a-dd5b-4741-9f66-4ad316f2b544" (UID: "3ba1a57a-dd5b-4741-9f66-4ad316f2b544"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:56:08.669916 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.669879 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-kube-api-access-nsj8m" (OuterVolumeSpecName: "kube-api-access-nsj8m") pod "3ba1a57a-dd5b-4741-9f66-4ad316f2b544" (UID: "3ba1a57a-dd5b-4741-9f66-4ad316f2b544"). InnerVolumeSpecName "kube-api-access-nsj8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:56:08.673255 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.673206 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-util" (OuterVolumeSpecName: "util") pod "3ba1a57a-dd5b-4741-9f66-4ad316f2b544" (UID: "3ba1a57a-dd5b-4741-9f66-4ad316f2b544"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:56:08.768350 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.768264 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:56:08.768350 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.768298 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:56:08.768350 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:08.768310 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsj8m\" (UniqueName: \"kubernetes.io/projected/3ba1a57a-dd5b-4741-9f66-4ad316f2b544-kube-api-access-nsj8m\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:56:09.458419 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:09.458383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" event={"ID":"3ba1a57a-dd5b-4741-9f66-4ad316f2b544","Type":"ContainerDied","Data":"0ffc576a818e15cd354c70b5df852e70a46698ea0ed9e46f19bd71979533dd35"} Jun 10 12:56:09.458880 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:09.458431 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ffc576a818e15cd354c70b5df852e70a46698ea0ed9e46f19bd71979533dd35" Jun 10 12:56:09.458880 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:09.458431 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2jq5w" Jun 10 12:56:10.464109 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:10.464073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" event={"ID":"d8afad34-7253-4f96-8b7b-2d575afbe902","Type":"ContainerStarted","Data":"80f70e64ab82e01c10b8e28bd781a68419b7d1386f234f530d546e47d2b55611"} Jun 10 12:56:10.481355 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:10.481308 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-2czqn" podStartSLOduration=1.134693243 podStartE2EDuration="3.48129248s" totalImagesPullingTime="2.346599237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:56:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:56:10.480951056 +0000 UTC m=+486.187255259" watchObservedRunningTime="2026-06-10 12:56:10.48129248 +0000 UTC m=+486.187596669" Jun 10 12:56:26.011423 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.011386 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f"] Jun 10 12:56:26.011939 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.011611 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerName="pull" Jun 10 12:56:26.011939 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.011622 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerName="pull" Jun 10 12:56:26.011939 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.011629 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerName="extract" Jun 10 12:56:26.011939 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.011635 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerName="extract" Jun 10 12:56:26.011939 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.011651 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerName="util" Jun 10 12:56:26.011939 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.011657 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerName="util" Jun 10 12:56:26.011939 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.011698 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba1a57a-dd5b-4741-9f66-4ad316f2b544" containerName="extract" Jun 10 12:56:26.014574 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.014558 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.017913 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.017892 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:56:26.019197 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.019179 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:56:26.019281 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.019179 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jcl4f\"" Jun 10 12:56:26.027697 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.027661 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f"] Jun 10 12:56:26.093534 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.093498 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.093534 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.093541 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.093744 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.093574 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmpfl\" (UniqueName: \"kubernetes.io/projected/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-kube-api-access-fmpfl\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.194994 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.194957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmpfl\" (UniqueName: \"kubernetes.io/projected/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-kube-api-access-fmpfl\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.195193 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.195011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.195193 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.195033 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.195379 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.195361 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.195469 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.195445 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.206547 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.206523 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmpfl\" (UniqueName: \"kubernetes.io/projected/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-kube-api-access-fmpfl\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.323477 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.323391 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:26.453022 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.452873 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f"] Jun 10 12:56:26.455505 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:56:26.455476 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef3a1fc_9143_4ec1_a1fe_9a0299bb7f3b.slice/crio-7f72fd0b0202ff18c921a659837e3676765bb7389871f0cad5e509fff61cdd4a WatchSource:0}: Error finding container 7f72fd0b0202ff18c921a659837e3676765bb7389871f0cad5e509fff61cdd4a: Status 404 returned error can't find the container with id 7f72fd0b0202ff18c921a659837e3676765bb7389871f0cad5e509fff61cdd4a Jun 10 12:56:26.521073 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.521046 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" event={"ID":"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b","Type":"ContainerStarted","Data":"8f6528078ce1a4aff4e7e36b390fd5365a1782ecb3617838f082a946cb98a7cf"} Jun 10 12:56:26.521189 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:26.521091 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" event={"ID":"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b","Type":"ContainerStarted","Data":"7f72fd0b0202ff18c921a659837e3676765bb7389871f0cad5e509fff61cdd4a"} Jun 10 12:56:27.525936 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:27.525846 2577 generic.go:356] "Generic (PLEG): container finished" podID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerID="8f6528078ce1a4aff4e7e36b390fd5365a1782ecb3617838f082a946cb98a7cf" exitCode=0 Jun 10 12:56:27.525936 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:27.525885 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" event={"ID":"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b","Type":"ContainerDied","Data":"8f6528078ce1a4aff4e7e36b390fd5365a1782ecb3617838f082a946cb98a7cf"} Jun 10 12:56:28.531450 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:28.531420 2577 generic.go:356] "Generic (PLEG): container finished" podID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerID="a5bc414017ed79d2b855c6c8bfeff1282430a9536e38260f7f824099427b481e" exitCode=0 Jun 10 12:56:28.531831 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:28.531479 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" event={"ID":"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b","Type":"ContainerDied","Data":"a5bc414017ed79d2b855c6c8bfeff1282430a9536e38260f7f824099427b481e"} Jun 10 12:56:29.536945 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:29.536913 2577 generic.go:356] "Generic (PLEG): container finished" podID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerID="d7699447b0e73cab2c6e1a7b8c6a38e15dbab25bb78d9645c549532a4bd866d1" exitCode=0 Jun 10 12:56:29.537324 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:29.536978 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" event={"ID":"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b","Type":"ContainerDied","Data":"d7699447b0e73cab2c6e1a7b8c6a38e15dbab25bb78d9645c549532a4bd866d1"} Jun 10 12:56:30.665535 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.665510 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:30.725608 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.725581 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-util\") pod \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " Jun 10 12:56:30.725788 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.725619 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmpfl\" (UniqueName: \"kubernetes.io/projected/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-kube-api-access-fmpfl\") pod \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " Jun 10 12:56:30.725788 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.725674 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-bundle\") pod \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\" (UID: \"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b\") " Jun 10 12:56:30.728228 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.726935 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-bundle" (OuterVolumeSpecName: "bundle") pod "aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" (UID: "aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:56:30.731753 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.731557 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-kube-api-access-fmpfl" (OuterVolumeSpecName: "kube-api-access-fmpfl") pod "aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" (UID: "aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b"). InnerVolumeSpecName "kube-api-access-fmpfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:56:30.733239 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.733206 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-util" (OuterVolumeSpecName: "util") pod "aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" (UID: "aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:56:30.827616 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.827583 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:56:30.827616 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.827612 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmpfl\" (UniqueName: \"kubernetes.io/projected/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-kube-api-access-fmpfl\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:56:30.827616 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:30.827623 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:56:31.545615 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:31.545578 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" event={"ID":"aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b","Type":"ContainerDied","Data":"7f72fd0b0202ff18c921a659837e3676765bb7389871f0cad5e509fff61cdd4a"} Jun 10 12:56:31.545790 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:31.545623 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdxxn5f" Jun 10 12:56:31.545790 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:31.545633 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f72fd0b0202ff18c921a659837e3676765bb7389871f0cad5e509fff61cdd4a" Jun 10 12:56:42.402521 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.402487 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q"] Jun 10 12:56:42.402977 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.402710 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerName="util" Jun 10 12:56:42.402977 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.402720 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerName="util" Jun 10 12:56:42.402977 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.402735 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerName="extract" Jun 10 12:56:42.402977 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.402740 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerName="extract" Jun 10 12:56:42.402977 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.402752 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerName="pull" Jun 10 12:56:42.402977 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.402758 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerName="pull" Jun 10 12:56:42.402977 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.402800 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aef3a1fc-9143-4ec1-a1fe-9a0299bb7f3b" containerName="extract" Jun 10 12:56:42.408796 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.408775 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.413231 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.413212 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 10 12:56:42.413526 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.413507 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 10 12:56:42.413612 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.413511 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 10 12:56:42.413612 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.413550 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vckhl\"" Jun 10 12:56:42.413891 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.413877 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 10 12:56:42.424790 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.424766 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q"] Jun 10 12:56:42.512223 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.512187 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efcab33d-1f6b-4a52-8141-f9228f2b83bc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.512223 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.512228 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efcab33d-1f6b-4a52-8141-f9228f2b83bc-tmp\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.512447 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.512298 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5wt6\" (UniqueName: \"kubernetes.io/projected/efcab33d-1f6b-4a52-8141-f9228f2b83bc-kube-api-access-b5wt6\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.512447 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.512386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/efcab33d-1f6b-4a52-8141-f9228f2b83bc-manifests\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.512447 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.512413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/efcab33d-1f6b-4a52-8141-f9228f2b83bc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.613508 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.613476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/efcab33d-1f6b-4a52-8141-f9228f2b83bc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.613699 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.613517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efcab33d-1f6b-4a52-8141-f9228f2b83bc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.613699 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.613549 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efcab33d-1f6b-4a52-8141-f9228f2b83bc-tmp\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.613699 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.613593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5wt6\" (UniqueName: \"kubernetes.io/projected/efcab33d-1f6b-4a52-8141-f9228f2b83bc-kube-api-access-b5wt6\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.613699 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.613652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/efcab33d-1f6b-4a52-8141-f9228f2b83bc-manifests\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.614077 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.614056 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/efcab33d-1f6b-4a52-8141-f9228f2b83bc-manifests\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.615817 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.615783 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efcab33d-1f6b-4a52-8141-f9228f2b83bc-tmp\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.616066 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.616046 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efcab33d-1f6b-4a52-8141-f9228f2b83bc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.616136 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.616117 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/efcab33d-1f6b-4a52-8141-f9228f2b83bc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.621681 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.621659 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5wt6\" (UniqueName: \"kubernetes.io/projected/efcab33d-1f6b-4a52-8141-f9228f2b83bc-kube-api-access-b5wt6\") pod \"opendatahub-operator-controller-manager-7f4c98d49d-7vk7q\" (UID: \"efcab33d-1f6b-4a52-8141-f9228f2b83bc\") " pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.718998 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.718905 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:42.852715 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:42.852689 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q"] Jun 10 12:56:42.855201 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:56:42.855174 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefcab33d_1f6b_4a52_8141_f9228f2b83bc.slice/crio-e1fa3ae76d41b2d286a947fcb8cbd7b077d63bf3a218f0d6d9b7d2d5bf08575c WatchSource:0}: Error finding container e1fa3ae76d41b2d286a947fcb8cbd7b077d63bf3a218f0d6d9b7d2d5bf08575c: Status 404 returned error can't find the container with id e1fa3ae76d41b2d286a947fcb8cbd7b077d63bf3a218f0d6d9b7d2d5bf08575c Jun 10 12:56:43.591620 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:43.591574 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" event={"ID":"efcab33d-1f6b-4a52-8141-f9228f2b83bc","Type":"ContainerStarted","Data":"e1fa3ae76d41b2d286a947fcb8cbd7b077d63bf3a218f0d6d9b7d2d5bf08575c"} Jun 10 12:56:46.605177 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:46.605140 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" event={"ID":"efcab33d-1f6b-4a52-8141-f9228f2b83bc","Type":"ContainerStarted","Data":"62e0fae43fe5e801cc0d5ac6bc50ad66e3b469abfddb9711c1a1c27dbf4aac8c"} Jun 10 12:56:47.610756 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:47.610720 2577 generic.go:356] "Generic (PLEG): container finished" podID="efcab33d-1f6b-4a52-8141-f9228f2b83bc" containerID="62e0fae43fe5e801cc0d5ac6bc50ad66e3b469abfddb9711c1a1c27dbf4aac8c" exitCode=0 Jun 10 12:56:47.611164 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:47.610828 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" event={"ID":"efcab33d-1f6b-4a52-8141-f9228f2b83bc","Type":"ContainerDied","Data":"62e0fae43fe5e801cc0d5ac6bc50ad66e3b469abfddb9711c1a1c27dbf4aac8c"} Jun 10 12:56:48.616552 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:48.616507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" event={"ID":"efcab33d-1f6b-4a52-8141-f9228f2b83bc","Type":"ContainerStarted","Data":"1530b7819f55431d2c80b9cae9e9cefbd5e95f2b0c12cb60cef1a908577fa1fe"} Jun 10 12:56:48.616983 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:48.616750 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:56:48.671605 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:48.671535 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" podStartSLOduration=2.529673296 podStartE2EDuration="6.671519336s" totalImagesPullingTime="3.194760651s" totalInitContainerRuntime="947.085389ms" isStatefulPod=false podCreationTimestamp="2026-06-10 12:56:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:56:48.664436399 +0000 UTC m=+524.370740588" watchObservedRunningTime="2026-06-10 12:56:48.671519336 +0000 UTC m=+524.377823524" Jun 10 12:56:59.622144 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:56:59.622113 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f4c98d49d-7vk7q" Jun 10 12:57:02.727339 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.727301 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-fz8sb"] Jun 10 12:57:02.733793 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.733773 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:02.736404 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.736383 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 10 12:57:02.736492 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.736420 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-swfqm\"" Jun 10 12:57:02.739656 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.739631 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-fz8sb"] Jun 10 12:57:02.886289 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.886255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1e0b919-f45a-444a-8631-703036d2bfce-cert\") pod \"odh-model-controller-6dc94f689f-fz8sb\" (UID: \"d1e0b919-f45a-444a-8631-703036d2bfce\") " pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:02.886468 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.886310 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb2dq\" (UniqueName: \"kubernetes.io/projected/d1e0b919-f45a-444a-8631-703036d2bfce-kube-api-access-mb2dq\") pod \"odh-model-controller-6dc94f689f-fz8sb\" (UID: \"d1e0b919-f45a-444a-8631-703036d2bfce\") " pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:02.987739 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.987658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1e0b919-f45a-444a-8631-703036d2bfce-cert\") pod \"odh-model-controller-6dc94f689f-fz8sb\" (UID: \"d1e0b919-f45a-444a-8631-703036d2bfce\") " pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:02.987739 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:02.987709 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb2dq\" (UniqueName: \"kubernetes.io/projected/d1e0b919-f45a-444a-8631-703036d2bfce-kube-api-access-mb2dq\") pod \"odh-model-controller-6dc94f689f-fz8sb\" (UID: \"d1e0b919-f45a-444a-8631-703036d2bfce\") " pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:02.987978 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:57:02.987848 2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 10 12:57:02.987978 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:57:02.987935 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e0b919-f45a-444a-8631-703036d2bfce-cert podName:d1e0b919-f45a-444a-8631-703036d2bfce nodeName:}" failed. No retries permitted until 2026-06-10 12:57:03.487912262 +0000 UTC m=+539.194216430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d1e0b919-f45a-444a-8631-703036d2bfce-cert") pod "odh-model-controller-6dc94f689f-fz8sb" (UID: "d1e0b919-f45a-444a-8631-703036d2bfce") : secret "odh-model-controller-webhook-cert" not found Jun 10 12:57:03.006270 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.006244 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb2dq\" (UniqueName: \"kubernetes.io/projected/d1e0b919-f45a-444a-8631-703036d2bfce-kube-api-access-mb2dq\") pod \"odh-model-controller-6dc94f689f-fz8sb\" (UID: \"d1e0b919-f45a-444a-8631-703036d2bfce\") " pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:03.383734 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.383699 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m"] Jun 10 12:57:03.387117 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.387100 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.390551 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.390530 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:57:03.391613 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.391594 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:57:03.391733 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.391599 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jcl4f\"" Jun 10 12:57:03.402814 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.402780 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m"] Jun 10 12:57:03.492599 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.492568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.492762 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.492615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1e0b919-f45a-444a-8631-703036d2bfce-cert\") pod \"odh-model-controller-6dc94f689f-fz8sb\" (UID: \"d1e0b919-f45a-444a-8631-703036d2bfce\") " pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:03.492762 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.492668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.492762 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.492733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k64t2\" (UniqueName: \"kubernetes.io/projected/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-kube-api-access-k64t2\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.494981 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.494955 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1e0b919-f45a-444a-8631-703036d2bfce-cert\") pod \"odh-model-controller-6dc94f689f-fz8sb\" (UID: \"d1e0b919-f45a-444a-8631-703036d2bfce\") " pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:03.504566 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.504543 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5796c65c65-jxm89"] Jun 10 12:57:03.507651 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.507630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.514910 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.514890 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 10 12:57:03.514910 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.514899 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 10 12:57:03.515117 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.515096 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4ckmj\"" Jun 10 12:57:03.515225 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.515186 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 10 12:57:03.515470 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.515456 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 10 12:57:03.593434 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.593399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8500ab8f-1dd1-4216-a61b-e3a28b91655b-tmp\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.593434 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.593440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k64t2\" (UniqueName: \"kubernetes.io/projected/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-kube-api-access-k64t2\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.593664 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.593513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.593664 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.593565 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8500ab8f-1dd1-4216-a61b-e3a28b91655b-tls-certs\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.593664 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.593601 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsb42\" (UniqueName: \"kubernetes.io/projected/8500ab8f-1dd1-4216-a61b-e3a28b91655b-kube-api-access-lsb42\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.593785 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.593664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.593962 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.593947 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.593997 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.593977 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.625551 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.625518 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k64t2\" (UniqueName: \"kubernetes.io/projected/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-kube-api-access-k64t2\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.628643 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.628618 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5796c65c65-jxm89"] Jun 10 12:57:03.645083 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.645020 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:03.695951 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.694574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsb42\" (UniqueName: \"kubernetes.io/projected/8500ab8f-1dd1-4216-a61b-e3a28b91655b-kube-api-access-lsb42\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.695951 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.694645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8500ab8f-1dd1-4216-a61b-e3a28b91655b-tmp\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.695951 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.694720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8500ab8f-1dd1-4216-a61b-e3a28b91655b-tls-certs\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.697017 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.696728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:03.698141 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.698088 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8500ab8f-1dd1-4216-a61b-e3a28b91655b-tmp\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.698249 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.698166 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8500ab8f-1dd1-4216-a61b-e3a28b91655b-tls-certs\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.705712 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.705662 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsb42\" (UniqueName: \"kubernetes.io/projected/8500ab8f-1dd1-4216-a61b-e3a28b91655b-kube-api-access-lsb42\") pod \"kube-auth-proxy-5796c65c65-jxm89\" (UID: \"8500ab8f-1dd1-4216-a61b-e3a28b91655b\") " pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.769399 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.769369 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-skxnl"] Jun 10 12:57:03.773544 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.773526 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:03.776874 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.776850 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 10 12:57:03.793159 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.793134 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-skxnl"] Jun 10 12:57:03.807533 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.807478 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-fz8sb"] Jun 10 12:57:03.810396 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:57:03.810370 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e0b919_f45a_444a_8631_703036d2bfce.slice/crio-be3380257f8620606c2ef119167cf7afb9c49b0cc07b74cdec20ab7cabcba08e WatchSource:0}: Error finding container be3380257f8620606c2ef119167cf7afb9c49b0cc07b74cdec20ab7cabcba08e: Status 404 returned error can't find the container with id be3380257f8620606c2ef119167cf7afb9c49b0cc07b74cdec20ab7cabcba08e Jun 10 12:57:03.817515 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.817290 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" Jun 10 12:57:03.844001 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.843974 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m"] Jun 10 12:57:03.844918 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:57:03.844878 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e26f2b_6012_4e12_8f1e_ebddf0523cb0.slice/crio-193b53480291d70a4882f2ccb9c1c74e15ed5163e88f97b4b64dd00a4f1a17d7 WatchSource:0}: Error finding container 193b53480291d70a4882f2ccb9c1c74e15ed5163e88f97b4b64dd00a4f1a17d7: Status 404 returned error can't find the container with id 193b53480291d70a4882f2ccb9c1c74e15ed5163e88f97b4b64dd00a4f1a17d7 Jun 10 12:57:03.896418 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.896333 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7nfq\" (UniqueName: \"kubernetes.io/projected/bbf88196-c7ea-4013-ba18-c17da3bce1ae-kube-api-access-j7nfq\") pod \"model-serving-api-5dd488fc48-skxnl\" (UID: \"bbf88196-c7ea-4013-ba18-c17da3bce1ae\") " pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:03.896418 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.896380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf88196-c7ea-4013-ba18-c17da3bce1ae-tls-certs\") pod \"model-serving-api-5dd488fc48-skxnl\" (UID: \"bbf88196-c7ea-4013-ba18-c17da3bce1ae\") " pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:03.960642 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.960610 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5796c65c65-jxm89"] Jun 10 12:57:03.964596 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:57:03.964567 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8500ab8f_1dd1_4216_a61b_e3a28b91655b.slice/crio-3befbdfa318ca3c56058bbea628d41aaae7b31d8c9908bdd8dfd0a675ff8cb4b WatchSource:0}: Error finding container 3befbdfa318ca3c56058bbea628d41aaae7b31d8c9908bdd8dfd0a675ff8cb4b: Status 404 returned error can't find the container with id 3befbdfa318ca3c56058bbea628d41aaae7b31d8c9908bdd8dfd0a675ff8cb4b Jun 10 12:57:03.997766 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.997740 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7nfq\" (UniqueName: \"kubernetes.io/projected/bbf88196-c7ea-4013-ba18-c17da3bce1ae-kube-api-access-j7nfq\") pod \"model-serving-api-5dd488fc48-skxnl\" (UID: \"bbf88196-c7ea-4013-ba18-c17da3bce1ae\") " pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:03.997909 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:03.997786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf88196-c7ea-4013-ba18-c17da3bce1ae-tls-certs\") pod \"model-serving-api-5dd488fc48-skxnl\" (UID: \"bbf88196-c7ea-4013-ba18-c17da3bce1ae\") " pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:03.997966 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:57:03.997917 2577 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 10 12:57:03.997999 ip-10-0-140-124 kubenswrapper[2577]: E0610 12:57:03.997968 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbf88196-c7ea-4013-ba18-c17da3bce1ae-tls-certs podName:bbf88196-c7ea-4013-ba18-c17da3bce1ae nodeName:}" failed. No retries permitted until 2026-06-10 12:57:04.497951783 +0000 UTC m=+540.204255950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bbf88196-c7ea-4013-ba18-c17da3bce1ae-tls-certs") pod "model-serving-api-5dd488fc48-skxnl" (UID: "bbf88196-c7ea-4013-ba18-c17da3bce1ae") : secret "model-serving-api-tls" not found Jun 10 12:57:04.007871 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.007848 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7nfq\" (UniqueName: \"kubernetes.io/projected/bbf88196-c7ea-4013-ba18-c17da3bce1ae-kube-api-access-j7nfq\") pod \"model-serving-api-5dd488fc48-skxnl\" (UID: \"bbf88196-c7ea-4013-ba18-c17da3bce1ae\") " pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:04.501704 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.501658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf88196-c7ea-4013-ba18-c17da3bce1ae-tls-certs\") pod \"model-serving-api-5dd488fc48-skxnl\" (UID: \"bbf88196-c7ea-4013-ba18-c17da3bce1ae\") " pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:04.504762 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.504706 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf88196-c7ea-4013-ba18-c17da3bce1ae-tls-certs\") pod \"model-serving-api-5dd488fc48-skxnl\" (UID: \"bbf88196-c7ea-4013-ba18-c17da3bce1ae\") " pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:04.683578 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.683318 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:04.684722 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.683993 2577 generic.go:356] "Generic (PLEG): container finished" podID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerID="4e30071a8f548508e84b7cfc96e51b5046fa313a600a48c9b198071293602c08" exitCode=0 Jun 10 12:57:04.684722 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.684079 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" event={"ID":"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0","Type":"ContainerDied","Data":"4e30071a8f548508e84b7cfc96e51b5046fa313a600a48c9b198071293602c08"} Jun 10 12:57:04.684722 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.684138 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" event={"ID":"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0","Type":"ContainerStarted","Data":"193b53480291d70a4882f2ccb9c1c74e15ed5163e88f97b4b64dd00a4f1a17d7"} Jun 10 12:57:04.689156 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.689066 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" event={"ID":"d1e0b919-f45a-444a-8631-703036d2bfce","Type":"ContainerStarted","Data":"be3380257f8620606c2ef119167cf7afb9c49b0cc07b74cdec20ab7cabcba08e"} Jun 10 12:57:04.691354 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.691295 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" event={"ID":"8500ab8f-1dd1-4216-a61b-e3a28b91655b","Type":"ContainerStarted","Data":"3befbdfa318ca3c56058bbea628d41aaae7b31d8c9908bdd8dfd0a675ff8cb4b"} Jun 10 12:57:04.871075 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:04.871036 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-skxnl"] Jun 10 12:57:04.874477 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:57:04.874301 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf88196_c7ea_4013_ba18_c17da3bce1ae.slice/crio-34d6046422f0be3abfd16c9c0c8fec420f313550f3bf89fbd5d84166ec0bdcdf WatchSource:0}: Error finding container 34d6046422f0be3abfd16c9c0c8fec420f313550f3bf89fbd5d84166ec0bdcdf: Status 404 returned error can't find the container with id 34d6046422f0be3abfd16c9c0c8fec420f313550f3bf89fbd5d84166ec0bdcdf Jun 10 12:57:05.700283 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:05.700203 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-skxnl" event={"ID":"bbf88196-c7ea-4013-ba18-c17da3bce1ae","Type":"ContainerStarted","Data":"34d6046422f0be3abfd16c9c0c8fec420f313550f3bf89fbd5d84166ec0bdcdf"} Jun 10 12:57:08.691978 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:08.691955 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 10 12:57:09.720058 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.720022 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" event={"ID":"8500ab8f-1dd1-4216-a61b-e3a28b91655b","Type":"ContainerStarted","Data":"cb6945285d05394f1528f586bf7a031d4b2e522187179fd0cfe01ff18c6d2647"} Jun 10 12:57:09.721580 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.721546 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-skxnl" event={"ID":"bbf88196-c7ea-4013-ba18-c17da3bce1ae","Type":"ContainerStarted","Data":"04a4ea6a1b28b4c1d43bc8ba76c178be2c2a10933e0dece97516511a557059a1"} Jun 10 12:57:09.721708 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.721604 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:09.723202 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.723180 2577 generic.go:356] "Generic (PLEG): container finished" podID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerID="43b8e8b5ed48e7a7e4288d3a8eb51f2f83d04b9b9de4dbb7b146d66c2114bf53" exitCode=0 Jun 10 12:57:09.723312 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.723237 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" event={"ID":"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0","Type":"ContainerDied","Data":"43b8e8b5ed48e7a7e4288d3a8eb51f2f83d04b9b9de4dbb7b146d66c2114bf53"} Jun 10 12:57:09.724736 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.724710 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" event={"ID":"d1e0b919-f45a-444a-8631-703036d2bfce","Type":"ContainerStarted","Data":"606ded755c3f688efc70ac39b5d90363af0474c65c3d0b693231ce3f5feb03e0"} Jun 10 12:57:09.724890 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.724860 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:09.852658 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.852603 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5796c65c65-jxm89" podStartSLOduration=2.130580394 podStartE2EDuration="6.852584496s" totalImagesPullingTime="4.722004102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:57:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:57:09.78372609 +0000 UTC m=+545.490030279" watchObservedRunningTime="2026-06-10 12:57:09.852584496 +0000 UTC m=+545.558888692" Jun 10 12:57:09.852821 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.852693 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-skxnl" podStartSLOduration=3.039672027 podStartE2EDuration="6.852687281s" totalImagesPullingTime="3.813015254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:57:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:57:09.851775986 +0000 UTC m=+545.558080175" watchObservedRunningTime="2026-06-10 12:57:09.852687281 +0000 UTC m=+545.558991470" Jun 10 12:57:09.930114 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:09.930061 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" podStartSLOduration=3.066810169 podStartE2EDuration="7.930045426s" totalImagesPullingTime="4.863235257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:57:09.925864816 +0000 UTC m=+545.632169004" watchObservedRunningTime="2026-06-10 12:57:09.930045426 +0000 UTC m=+545.636349614" Jun 10 12:57:10.732775 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:10.732744 2577 generic.go:356] "Generic (PLEG): container finished" podID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerID="4760d3c99589e71183a6e86457a702bcd497d208705d72c2be5a05b9a0cdaa0d" exitCode=0 Jun 10 12:57:10.733205 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:10.732835 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" event={"ID":"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0","Type":"ContainerDied","Data":"4760d3c99589e71183a6e86457a702bcd497d208705d72c2be5a05b9a0cdaa0d"} Jun 10 12:57:11.862287 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:11.862238 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:11.977089 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:11.977062 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k64t2\" (UniqueName: \"kubernetes.io/projected/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-kube-api-access-k64t2\") pod \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " Jun 10 12:57:11.977227 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:11.977107 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-bundle\") pod \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " Jun 10 12:57:11.977227 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:11.977194 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-util\") pod \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\" (UID: \"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0\") " Jun 10 12:57:11.978005 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:11.977978 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-bundle" (OuterVolumeSpecName: "bundle") pod "d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" (UID: "d4e26f2b-6012-4e12-8f1e-ebddf0523cb0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:57:11.979297 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:11.979271 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-kube-api-access-k64t2" (OuterVolumeSpecName: "kube-api-access-k64t2") pod "d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" (UID: "d4e26f2b-6012-4e12-8f1e-ebddf0523cb0"). InnerVolumeSpecName "kube-api-access-k64t2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:57:11.981775 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:11.981755 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-util" (OuterVolumeSpecName: "util") pod "d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" (UID: "d4e26f2b-6012-4e12-8f1e-ebddf0523cb0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:57:12.078127 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:12.078036 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:57:12.078127 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:12.078070 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k64t2\" (UniqueName: \"kubernetes.io/projected/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-kube-api-access-k64t2\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:57:12.078127 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:12.078084 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4e26f2b-6012-4e12-8f1e-ebddf0523cb0-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:57:12.744068 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:12.744046 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" Jun 10 12:57:12.744217 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:12.744046 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab0699q2m" event={"ID":"d4e26f2b-6012-4e12-8f1e-ebddf0523cb0","Type":"ContainerDied","Data":"193b53480291d70a4882f2ccb9c1c74e15ed5163e88f97b4b64dd00a4f1a17d7"} Jun 10 12:57:12.744217 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:12.744156 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="193b53480291d70a4882f2ccb9c1c74e15ed5163e88f97b4b64dd00a4f1a17d7" Jun 10 12:57:13.180595 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.180564 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq"] Jun 10 12:57:13.181027 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.180854 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerName="extract" Jun 10 12:57:13.181027 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.180866 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerName="extract" Jun 10 12:57:13.181027 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.180875 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerName="pull" Jun 10 12:57:13.181027 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.180881 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerName="pull" Jun 10 12:57:13.181027 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.180895 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerName="util" Jun 10 12:57:13.181027 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.180900 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerName="util" Jun 10 12:57:13.181027 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.180940 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4e26f2b-6012-4e12-8f1e-ebddf0523cb0" containerName="extract" Jun 10 12:57:13.186080 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.186050 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.191140 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.191116 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-l2tb5\"" Jun 10 12:57:13.191286 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.191212 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 10 12:57:13.191355 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.191322 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 10 12:57:13.191813 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.191782 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 10 12:57:13.192100 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.192082 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:57:13.194059 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.194042 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 10 12:57:13.203873 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.203853 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq"] Jun 10 12:57:13.288327 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.288292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f5937b-fb14-4a7a-a4d4-c256390ea742-cert\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.288327 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.288330 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/07f5937b-fb14-4a7a-a4d4-c256390ea742-manager-config\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.288553 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.288352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/07f5937b-fb14-4a7a-a4d4-c256390ea742-metrics-cert\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.288553 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.288373 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk9wh\" (UniqueName: \"kubernetes.io/projected/07f5937b-fb14-4a7a-a4d4-c256390ea742-kube-api-access-sk9wh\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.389552 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.389517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f5937b-fb14-4a7a-a4d4-c256390ea742-cert\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.389552 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.389553 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/07f5937b-fb14-4a7a-a4d4-c256390ea742-manager-config\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.389778 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.389572 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/07f5937b-fb14-4a7a-a4d4-c256390ea742-metrics-cert\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.389778 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.389593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk9wh\" (UniqueName: \"kubernetes.io/projected/07f5937b-fb14-4a7a-a4d4-c256390ea742-kube-api-access-sk9wh\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.390365 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.390337 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/07f5937b-fb14-4a7a-a4d4-c256390ea742-manager-config\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.392068 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.392045 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f5937b-fb14-4a7a-a4d4-c256390ea742-cert\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.392158 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.392067 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/07f5937b-fb14-4a7a-a4d4-c256390ea742-metrics-cert\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.403842 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.403799 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk9wh\" (UniqueName: \"kubernetes.io/projected/07f5937b-fb14-4a7a-a4d4-c256390ea742-kube-api-access-sk9wh\") pod \"lws-controller-manager-546fcb9487-7rdgq\" (UID: \"07f5937b-fb14-4a7a-a4d4-c256390ea742\") " pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.496144 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.496056 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:13.646387 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.646269 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq"] Jun 10 12:57:13.648999 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:57:13.648955 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f5937b_fb14_4a7a_a4d4_c256390ea742.slice/crio-8494ddfcd3282d655a006284db3ceebb6bc07349adfdd3ee72dab8fc5091bbb9 WatchSource:0}: Error finding container 8494ddfcd3282d655a006284db3ceebb6bc07349adfdd3ee72dab8fc5091bbb9: Status 404 returned error can't find the container with id 8494ddfcd3282d655a006284db3ceebb6bc07349adfdd3ee72dab8fc5091bbb9 Jun 10 12:57:13.749567 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:13.749476 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" event={"ID":"07f5937b-fb14-4a7a-a4d4-c256390ea742","Type":"ContainerStarted","Data":"8494ddfcd3282d655a006284db3ceebb6bc07349adfdd3ee72dab8fc5091bbb9"} Jun 10 12:57:17.768061 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:17.768022 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" event={"ID":"07f5937b-fb14-4a7a-a4d4-c256390ea742","Type":"ContainerStarted","Data":"0965a0db291e159ac4e2b4b24a4f37f008bcedbc97b51a9e02ad3fd8b4732009"} Jun 10 12:57:17.768458 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:17.768117 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:57:17.787419 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:17.787361 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" podStartSLOduration=1.544904078 podStartE2EDuration="4.787345895s" totalImagesPullingTime="3.242441817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:57:17.784989255 +0000 UTC m=+553.491293444" watchObservedRunningTime="2026-06-10 12:57:17.787345895 +0000 UTC m=+553.493650087" Jun 10 12:57:19.556305 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.556269 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz"] Jun 10 12:57:19.567654 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.567630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.570660 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.570635 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:57:19.570902 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.570734 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jcl4f\"" Jun 10 12:57:19.571742 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.571722 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:57:19.581148 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.581124 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz"] Jun 10 12:57:19.627711 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.627681 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.627898 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.627717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.627898 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.627739 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdssm\" (UniqueName: \"kubernetes.io/projected/605e32a6-8830-4386-93cc-3b8f0512b309-kube-api-access-gdssm\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.728204 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.728174 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.728204 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.728210 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.728458 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.728230 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdssm\" (UniqueName: \"kubernetes.io/projected/605e32a6-8830-4386-93cc-3b8f0512b309-kube-api-access-gdssm\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.728556 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.728534 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.728625 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.728604 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.745691 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.745657 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdssm\" (UniqueName: \"kubernetes.io/projected/605e32a6-8830-4386-93cc-3b8f0512b309-kube-api-access-gdssm\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:19.876482 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:19.876451 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:20.082723 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:20.082678 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz"] Jun 10 12:57:20.087788 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:57:20.087758 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605e32a6_8830_4386_93cc_3b8f0512b309.slice/crio-3e04f0c19674bf51b87c4ebf1365e7e5bc2a9716d66858ff4525a10853d8215b WatchSource:0}: Error finding container 3e04f0c19674bf51b87c4ebf1365e7e5bc2a9716d66858ff4525a10853d8215b: Status 404 returned error can't find the container with id 3e04f0c19674bf51b87c4ebf1365e7e5bc2a9716d66858ff4525a10853d8215b Jun 10 12:57:20.735152 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:20.735123 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-fz8sb" Jun 10 12:57:20.737530 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:20.737507 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-skxnl" Jun 10 12:57:20.783256 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:20.783223 2577 generic.go:356] "Generic (PLEG): container finished" podID="605e32a6-8830-4386-93cc-3b8f0512b309" containerID="7b4694fb397bf5105e49695e649e77c2c716d15feb9f2461efa1b13d321fdb10" exitCode=0 Jun 10 12:57:20.783410 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:20.783300 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" event={"ID":"605e32a6-8830-4386-93cc-3b8f0512b309","Type":"ContainerDied","Data":"7b4694fb397bf5105e49695e649e77c2c716d15feb9f2461efa1b13d321fdb10"} Jun 10 12:57:20.783410 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:20.783361 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" event={"ID":"605e32a6-8830-4386-93cc-3b8f0512b309","Type":"ContainerStarted","Data":"3e04f0c19674bf51b87c4ebf1365e7e5bc2a9716d66858ff4525a10853d8215b"} Jun 10 12:57:22.793105 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:22.793072 2577 generic.go:356] "Generic (PLEG): container finished" podID="605e32a6-8830-4386-93cc-3b8f0512b309" containerID="801050e9a1a1f5023bcb842b968225f51395bc5c319af0eb509ff70a5e7350ac" exitCode=0 Jun 10 12:57:22.793487 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:22.793132 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" event={"ID":"605e32a6-8830-4386-93cc-3b8f0512b309","Type":"ContainerDied","Data":"801050e9a1a1f5023bcb842b968225f51395bc5c319af0eb509ff70a5e7350ac"} Jun 10 12:57:23.799463 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:23.799428 2577 generic.go:356] "Generic (PLEG): container finished" podID="605e32a6-8830-4386-93cc-3b8f0512b309" containerID="fefa3a6029fe8a499b0664bfadd160604a423d9b9246f1c738e949825b5e0512" exitCode=0 Jun 10 12:57:23.799860 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:23.799494 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" event={"ID":"605e32a6-8830-4386-93cc-3b8f0512b309","Type":"ContainerDied","Data":"fefa3a6029fe8a499b0664bfadd160604a423d9b9246f1c738e949825b5e0512"} Jun 10 12:57:24.928937 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:24.928908 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:24.962468 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:24.962441 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-bundle\") pod \"605e32a6-8830-4386-93cc-3b8f0512b309\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " Jun 10 12:57:24.962655 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:24.962477 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-util\") pod \"605e32a6-8830-4386-93cc-3b8f0512b309\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " Jun 10 12:57:24.962655 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:24.962517 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdssm\" (UniqueName: \"kubernetes.io/projected/605e32a6-8830-4386-93cc-3b8f0512b309-kube-api-access-gdssm\") pod \"605e32a6-8830-4386-93cc-3b8f0512b309\" (UID: \"605e32a6-8830-4386-93cc-3b8f0512b309\") " Jun 10 12:57:24.963506 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:24.963474 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-bundle" (OuterVolumeSpecName: "bundle") pod "605e32a6-8830-4386-93cc-3b8f0512b309" (UID: "605e32a6-8830-4386-93cc-3b8f0512b309"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:57:24.964640 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:24.964615 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/605e32a6-8830-4386-93cc-3b8f0512b309-kube-api-access-gdssm" (OuterVolumeSpecName: "kube-api-access-gdssm") pod "605e32a6-8830-4386-93cc-3b8f0512b309" (UID: "605e32a6-8830-4386-93cc-3b8f0512b309"). InnerVolumeSpecName "kube-api-access-gdssm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:57:24.970486 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:24.970453 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-util" (OuterVolumeSpecName: "util") pod "605e32a6-8830-4386-93cc-3b8f0512b309" (UID: "605e32a6-8830-4386-93cc-3b8f0512b309"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:57:25.063436 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:25.063343 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:57:25.063436 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:25.063374 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605e32a6-8830-4386-93cc-3b8f0512b309-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:57:25.063436 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:25.063386 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdssm\" (UniqueName: \"kubernetes.io/projected/605e32a6-8830-4386-93cc-3b8f0512b309-kube-api-access-gdssm\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:57:25.809205 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:25.809179 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" Jun 10 12:57:25.809205 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:25.809199 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762agcdjz" event={"ID":"605e32a6-8830-4386-93cc-3b8f0512b309","Type":"ContainerDied","Data":"3e04f0c19674bf51b87c4ebf1365e7e5bc2a9716d66858ff4525a10853d8215b"} Jun 10 12:57:25.809485 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:25.809242 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e04f0c19674bf51b87c4ebf1365e7e5bc2a9716d66858ff4525a10853d8215b" Jun 10 12:57:28.775454 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:57:28.775421 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-546fcb9487-7rdgq" Jun 10 12:58:04.767054 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:04.767025 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:58:04.767626 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:04.767607 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 12:58:19.162935 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.162849 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw"] Jun 10 12:58:19.165358 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.163119 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605e32a6-8830-4386-93cc-3b8f0512b309" containerName="pull" Jun 10 12:58:19.165358 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.163129 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="605e32a6-8830-4386-93cc-3b8f0512b309" containerName="pull" Jun 10 12:58:19.165358 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.163144 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605e32a6-8830-4386-93cc-3b8f0512b309" containerName="util" Jun 10 12:58:19.165358 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.163150 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="605e32a6-8830-4386-93cc-3b8f0512b309" containerName="util" Jun 10 12:58:19.165358 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.163157 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605e32a6-8830-4386-93cc-3b8f0512b309" containerName="extract" Jun 10 12:58:19.165358 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.163162 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="605e32a6-8830-4386-93cc-3b8f0512b309" containerName="extract" Jun 10 12:58:19.165358 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.163210 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="605e32a6-8830-4386-93cc-3b8f0512b309" containerName="extract" Jun 10 12:58:19.166192 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.166176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.170661 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.170640 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 10 12:58:19.170773 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.170656 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7xcnb\"" Jun 10 12:58:19.170773 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.170700 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 10 12:58:19.176441 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.176419 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw"] Jun 10 12:58:19.301240 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.301211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.301422 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.301265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtxf7\" (UniqueName: \"kubernetes.io/projected/b613471e-a5ab-4552-9d9d-387564bd9280-kube-api-access-mtxf7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.301422 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.301292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.402181 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.402137 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.402378 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.402209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtxf7\" (UniqueName: \"kubernetes.io/projected/b613471e-a5ab-4552-9d9d-387564bd9280-kube-api-access-mtxf7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.402378 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.402245 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.402546 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.402527 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.402609 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.402590 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.411301 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.411272 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtxf7\" (UniqueName: \"kubernetes.io/projected/b613471e-a5ab-4552-9d9d-387564bd9280-kube-api-access-mtxf7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.475171 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.475076 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:19.605482 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.605461 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw"] Jun 10 12:58:19.607612 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:58:19.607587 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb613471e_a5ab_4552_9d9d_387564bd9280.slice/crio-176b1be08440aeb254aa7397ce6b70f9acc29704efe6d3be338ce0c5fa8443e0 WatchSource:0}: Error finding container 176b1be08440aeb254aa7397ce6b70f9acc29704efe6d3be338ce0c5fa8443e0: Status 404 returned error can't find the container with id 176b1be08440aeb254aa7397ce6b70f9acc29704efe6d3be338ce0c5fa8443e0 Jun 10 12:58:19.974923 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.974891 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d"] Jun 10 12:58:19.978030 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.978014 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:19.990071 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:19.990046 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d"] Jun 10 12:58:20.056188 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.056153 2577 generic.go:356] "Generic (PLEG): container finished" podID="b613471e-a5ab-4552-9d9d-387564bd9280" containerID="ba080caec28bb7ef1b9a8a6c2a5ad0e753de01fbdfcab5845324e4f4da41a4c9" exitCode=0 Jun 10 12:58:20.056355 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.056207 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" event={"ID":"b613471e-a5ab-4552-9d9d-387564bd9280","Type":"ContainerDied","Data":"ba080caec28bb7ef1b9a8a6c2a5ad0e753de01fbdfcab5845324e4f4da41a4c9"} Jun 10 12:58:20.056355 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.056257 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" event={"ID":"b613471e-a5ab-4552-9d9d-387564bd9280","Type":"ContainerStarted","Data":"176b1be08440aeb254aa7397ce6b70f9acc29704efe6d3be338ce0c5fa8443e0"} Jun 10 12:58:20.056698 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.056684 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:58:20.107824 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.107774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.108006 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.107835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmt2x\" (UniqueName: \"kubernetes.io/projected/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-kube-api-access-bmt2x\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.108006 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.107887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.208498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.208449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.208498 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.208501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmt2x\" (UniqueName: \"kubernetes.io/projected/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-kube-api-access-bmt2x\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.209011 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.208541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.209011 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.208886 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.209011 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.208900 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.217711 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.217689 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmt2x\" (UniqueName: \"kubernetes.io/projected/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-kube-api-access-bmt2x\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.287628 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.287539 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:20.421139 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.421111 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d"] Jun 10 12:58:20.422242 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:58:20.422219 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07091df_a21f_4e5f_bb49_6e9ed46c25c1.slice/crio-25e7e177f8cef9d1a16933e971638b6f6b430befcce66a98b569c74e6dc391d9 WatchSource:0}: Error finding container 25e7e177f8cef9d1a16933e971638b6f6b430befcce66a98b569c74e6dc391d9: Status 404 returned error can't find the container with id 25e7e177f8cef9d1a16933e971638b6f6b430befcce66a98b569c74e6dc391d9 Jun 10 12:58:20.754252 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.754221 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj"] Jun 10 12:58:20.758111 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.758093 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:20.766330 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.766299 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj"] Jun 10 12:58:20.913057 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.913031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:20.913231 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.913065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnhqr\" (UniqueName: \"kubernetes.io/projected/2ac72622-74c6-43f0-8796-700350e39b7a-kube-api-access-fnhqr\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:20.913231 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:20.913091 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:21.014531 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.014434 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:21.014531 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.014486 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnhqr\" (UniqueName: \"kubernetes.io/projected/2ac72622-74c6-43f0-8796-700350e39b7a-kube-api-access-fnhqr\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:21.014531 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.014512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:21.014890 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.014864 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:21.014963 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.014916 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:21.023657 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.023634 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnhqr\" (UniqueName: \"kubernetes.io/projected/2ac72622-74c6-43f0-8796-700350e39b7a-kube-api-access-fnhqr\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:21.061568 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.061535 2577 generic.go:356] "Generic (PLEG): container finished" podID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerID="3cb8b7f8b5ac8d5ae341b65075076e5086e832e6af85e26da57bbc4a84c89e32" exitCode=0 Jun 10 12:58:21.061702 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.061615 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" event={"ID":"f07091df-a21f-4e5f-bb49-6e9ed46c25c1","Type":"ContainerDied","Data":"3cb8b7f8b5ac8d5ae341b65075076e5086e832e6af85e26da57bbc4a84c89e32"} Jun 10 12:58:21.061702 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.061666 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" event={"ID":"f07091df-a21f-4e5f-bb49-6e9ed46c25c1","Type":"ContainerStarted","Data":"25e7e177f8cef9d1a16933e971638b6f6b430befcce66a98b569c74e6dc391d9"} Jun 10 12:58:21.063557 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.063534 2577 generic.go:356] "Generic (PLEG): container finished" podID="b613471e-a5ab-4552-9d9d-387564bd9280" containerID="c06e3e25c7ee64f46c42b98bdfa3b7d672985bc9e73c20af4debea7ca676ed6f" exitCode=0 Jun 10 12:58:21.063651 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.063572 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" event={"ID":"b613471e-a5ab-4552-9d9d-387564bd9280","Type":"ContainerDied","Data":"c06e3e25c7ee64f46c42b98bdfa3b7d672985bc9e73c20af4debea7ca676ed6f"} Jun 10 12:58:21.069531 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.069517 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:21.177528 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.177496 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb"] Jun 10 12:58:21.181126 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.181098 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.193369 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.193343 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb"] Jun 10 12:58:21.209453 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.209418 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj"] Jun 10 12:58:21.254612 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:58:21.254583 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac72622_74c6_43f0_8796_700350e39b7a.slice/crio-3eaee86cd254c3a3f1860a3da26f8341bbc4e6693b9d64c90a2804f5fbbc0bc7 WatchSource:0}: Error finding container 3eaee86cd254c3a3f1860a3da26f8341bbc4e6693b9d64c90a2804f5fbbc0bc7: Status 404 returned error can't find the container with id 3eaee86cd254c3a3f1860a3da26f8341bbc4e6693b9d64c90a2804f5fbbc0bc7 Jun 10 12:58:21.316142 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.316112 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.316261 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.316149 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.316261 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.316168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbvtg\" (UniqueName: \"kubernetes.io/projected/f7c0372d-987d-4711-bbbe-bb126c2d90dd-kube-api-access-hbvtg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.417132 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.417100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.417312 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.417148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.417312 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.417171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbvtg\" (UniqueName: \"kubernetes.io/projected/f7c0372d-987d-4711-bbbe-bb126c2d90dd-kube-api-access-hbvtg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.417518 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.417498 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.417578 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.417523 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.426635 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.426605 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbvtg\" (UniqueName: \"kubernetes.io/projected/f7c0372d-987d-4711-bbbe-bb126c2d90dd-kube-api-access-hbvtg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.492519 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.492484 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:21.631827 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:21.631784 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb"] Jun 10 12:58:21.632920 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:58:21.632896 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c0372d_987d_4711_bbbe_bb126c2d90dd.slice/crio-f582974b8bc2a90444b5973e26d1a81ee5183a67771052afe2438b89b9e06fc7 WatchSource:0}: Error finding container f582974b8bc2a90444b5973e26d1a81ee5183a67771052afe2438b89b9e06fc7: Status 404 returned error can't find the container with id f582974b8bc2a90444b5973e26d1a81ee5183a67771052afe2438b89b9e06fc7 Jun 10 12:58:22.071337 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.071242 2577 generic.go:356] "Generic (PLEG): container finished" podID="b613471e-a5ab-4552-9d9d-387564bd9280" containerID="34c54ff92aa23cdb9f6c39b48e81e48db566ad9cb6a9371a3ddc1d12ccca77e4" exitCode=0 Jun 10 12:58:22.071493 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.071354 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" event={"ID":"b613471e-a5ab-4552-9d9d-387564bd9280","Type":"ContainerDied","Data":"34c54ff92aa23cdb9f6c39b48e81e48db566ad9cb6a9371a3ddc1d12ccca77e4"} Jun 10 12:58:22.073305 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.073276 2577 generic.go:356] "Generic (PLEG): container finished" podID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerID="d526e0cce0b1b1b31863b4890577de00a224a02dcb7f343913e8fcda365b0e41" exitCode=0 Jun 10 12:58:22.073395 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.073369 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" event={"ID":"f07091df-a21f-4e5f-bb49-6e9ed46c25c1","Type":"ContainerDied","Data":"d526e0cce0b1b1b31863b4890577de00a224a02dcb7f343913e8fcda365b0e41"} Jun 10 12:58:22.075026 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.074891 2577 generic.go:356] "Generic (PLEG): container finished" podID="2ac72622-74c6-43f0-8796-700350e39b7a" containerID="a548b01a8d0743d33f4f64b8748836666e1df4421e497bcdfc70bc8d80f3f701" exitCode=0 Jun 10 12:58:22.075026 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.074966 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" event={"ID":"2ac72622-74c6-43f0-8796-700350e39b7a","Type":"ContainerDied","Data":"a548b01a8d0743d33f4f64b8748836666e1df4421e497bcdfc70bc8d80f3f701"} Jun 10 12:58:22.075026 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.075014 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" event={"ID":"2ac72622-74c6-43f0-8796-700350e39b7a","Type":"ContainerStarted","Data":"3eaee86cd254c3a3f1860a3da26f8341bbc4e6693b9d64c90a2804f5fbbc0bc7"} Jun 10 12:58:22.076755 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.076732 2577 generic.go:356] "Generic (PLEG): container finished" podID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerID="9a3a061a4684cd4ef0de946d7ce9d1c5dcffa610f38609ed75d14269594552b3" exitCode=0 Jun 10 12:58:22.076886 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.076783 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" event={"ID":"f7c0372d-987d-4711-bbbe-bb126c2d90dd","Type":"ContainerDied","Data":"9a3a061a4684cd4ef0de946d7ce9d1c5dcffa610f38609ed75d14269594552b3"} Jun 10 12:58:22.076956 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:22.076893 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" event={"ID":"f7c0372d-987d-4711-bbbe-bb126c2d90dd","Type":"ContainerStarted","Data":"f582974b8bc2a90444b5973e26d1a81ee5183a67771052afe2438b89b9e06fc7"} Jun 10 12:58:23.083876 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.083766 2577 generic.go:356] "Generic (PLEG): container finished" podID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerID="5dc624ab54b48dfaf6591c61c1e094f4de7bb789bef5f8f0c7898394a83d93f1" exitCode=0 Jun 10 12:58:23.083876 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.083853 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" event={"ID":"f07091df-a21f-4e5f-bb49-6e9ed46c25c1","Type":"ContainerDied","Data":"5dc624ab54b48dfaf6591c61c1e094f4de7bb789bef5f8f0c7898394a83d93f1"} Jun 10 12:58:23.085604 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.085583 2577 generic.go:356] "Generic (PLEG): container finished" podID="2ac72622-74c6-43f0-8796-700350e39b7a" containerID="af466576f6f57d1f0faf04dea19e254899d133496c0add5227703e9df6a81152" exitCode=0 Jun 10 12:58:23.085706 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.085663 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" event={"ID":"2ac72622-74c6-43f0-8796-700350e39b7a","Type":"ContainerDied","Data":"af466576f6f57d1f0faf04dea19e254899d133496c0add5227703e9df6a81152"} Jun 10 12:58:23.087533 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.087508 2577 generic.go:356] "Generic (PLEG): container finished" podID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerID="f56a7c015e291271c21b1d5b6aba0d03d684b9adf45044491b3516bbe9487320" exitCode=0 Jun 10 12:58:23.087630 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.087538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" event={"ID":"f7c0372d-987d-4711-bbbe-bb126c2d90dd","Type":"ContainerDied","Data":"f56a7c015e291271c21b1d5b6aba0d03d684b9adf45044491b3516bbe9487320"} Jun 10 12:58:23.217470 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.217443 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:23.234499 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.234474 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-bundle\") pod \"b613471e-a5ab-4552-9d9d-387564bd9280\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " Jun 10 12:58:23.234618 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.234510 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-util\") pod \"b613471e-a5ab-4552-9d9d-387564bd9280\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " Jun 10 12:58:23.234618 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.234553 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtxf7\" (UniqueName: \"kubernetes.io/projected/b613471e-a5ab-4552-9d9d-387564bd9280-kube-api-access-mtxf7\") pod \"b613471e-a5ab-4552-9d9d-387564bd9280\" (UID: \"b613471e-a5ab-4552-9d9d-387564bd9280\") " Jun 10 12:58:23.234954 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.234925 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-bundle" (OuterVolumeSpecName: "bundle") pod "b613471e-a5ab-4552-9d9d-387564bd9280" (UID: "b613471e-a5ab-4552-9d9d-387564bd9280"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:58:23.236830 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.236788 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b613471e-a5ab-4552-9d9d-387564bd9280-kube-api-access-mtxf7" (OuterVolumeSpecName: "kube-api-access-mtxf7") pod "b613471e-a5ab-4552-9d9d-387564bd9280" (UID: "b613471e-a5ab-4552-9d9d-387564bd9280"). InnerVolumeSpecName "kube-api-access-mtxf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:58:23.243183 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.243149 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-util" (OuterVolumeSpecName: "util") pod "b613471e-a5ab-4552-9d9d-387564bd9280" (UID: "b613471e-a5ab-4552-9d9d-387564bd9280"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:58:23.335627 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.335540 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtxf7\" (UniqueName: \"kubernetes.io/projected/b613471e-a5ab-4552-9d9d-387564bd9280-kube-api-access-mtxf7\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:23.335627 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.335572 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:23.335627 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:23.335586 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b613471e-a5ab-4552-9d9d-387564bd9280-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:24.094235 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.094197 2577 generic.go:356] "Generic (PLEG): container finished" podID="2ac72622-74c6-43f0-8796-700350e39b7a" containerID="4db6c0d80c8c11faa8f75ea24ad70e1816bc9ca2fcba08265fb085849621ca1e" exitCode=0 Jun 10 12:58:24.094655 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.094283 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" event={"ID":"2ac72622-74c6-43f0-8796-700350e39b7a","Type":"ContainerDied","Data":"4db6c0d80c8c11faa8f75ea24ad70e1816bc9ca2fcba08265fb085849621ca1e"} Jun 10 12:58:24.096315 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.096296 2577 generic.go:356] "Generic (PLEG): container finished" podID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerID="74192b9e095ee2afe85b929ab607f6a26a60658f12051d945bef30349cec9d78" exitCode=0 Jun 10 12:58:24.096385 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.096326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" event={"ID":"f7c0372d-987d-4711-bbbe-bb126c2d90dd","Type":"ContainerDied","Data":"74192b9e095ee2afe85b929ab607f6a26a60658f12051d945bef30349cec9d78"} Jun 10 12:58:24.098203 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.098182 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" event={"ID":"b613471e-a5ab-4552-9d9d-387564bd9280","Type":"ContainerDied","Data":"176b1be08440aeb254aa7397ce6b70f9acc29704efe6d3be338ce0c5fa8443e0"} Jun 10 12:58:24.098311 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.098217 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="176b1be08440aeb254aa7397ce6b70f9acc29704efe6d3be338ce0c5fa8443e0" Jun 10 12:58:24.098311 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.098216 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw" Jun 10 12:58:24.226110 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.226088 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:24.240219 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.240195 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-bundle\") pod \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " Jun 10 12:58:24.240338 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.240251 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-util\") pod \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " Jun 10 12:58:24.240338 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.240287 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmt2x\" (UniqueName: \"kubernetes.io/projected/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-kube-api-access-bmt2x\") pod \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\" (UID: \"f07091df-a21f-4e5f-bb49-6e9ed46c25c1\") " Jun 10 12:58:24.240843 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.240786 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-bundle" (OuterVolumeSpecName: "bundle") pod "f07091df-a21f-4e5f-bb49-6e9ed46c25c1" (UID: "f07091df-a21f-4e5f-bb49-6e9ed46c25c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:58:24.242439 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.242414 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-kube-api-access-bmt2x" (OuterVolumeSpecName: "kube-api-access-bmt2x") pod "f07091df-a21f-4e5f-bb49-6e9ed46c25c1" (UID: "f07091df-a21f-4e5f-bb49-6e9ed46c25c1"). InnerVolumeSpecName "kube-api-access-bmt2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:58:24.248894 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.248867 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-util" (OuterVolumeSpecName: "util") pod "f07091df-a21f-4e5f-bb49-6e9ed46c25c1" (UID: "f07091df-a21f-4e5f-bb49-6e9ed46c25c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:58:24.341375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.341339 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmt2x\" (UniqueName: \"kubernetes.io/projected/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-kube-api-access-bmt2x\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:24.341375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.341366 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:24.341375 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:24.341376 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f07091df-a21f-4e5f-bb49-6e9ed46c25c1-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:25.104676 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.104592 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" Jun 10 12:58:25.104676 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.104624 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d" event={"ID":"f07091df-a21f-4e5f-bb49-6e9ed46c25c1","Type":"ContainerDied","Data":"25e7e177f8cef9d1a16933e971638b6f6b430befcce66a98b569c74e6dc391d9"} Jun 10 12:58:25.104676 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.104676 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25e7e177f8cef9d1a16933e971638b6f6b430befcce66a98b569c74e6dc391d9" Jun 10 12:58:25.229497 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.229474 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:25.246078 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.246057 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-bundle\") pod \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " Jun 10 12:58:25.246188 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.246111 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-util\") pod \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " Jun 10 12:58:25.246188 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.246147 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbvtg\" (UniqueName: \"kubernetes.io/projected/f7c0372d-987d-4711-bbbe-bb126c2d90dd-kube-api-access-hbvtg\") pod \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\" (UID: \"f7c0372d-987d-4711-bbbe-bb126c2d90dd\") " Jun 10 12:58:25.246613 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.246588 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-bundle" (OuterVolumeSpecName: "bundle") pod "f7c0372d-987d-4711-bbbe-bb126c2d90dd" (UID: "f7c0372d-987d-4711-bbbe-bb126c2d90dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:58:25.248374 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.248344 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c0372d-987d-4711-bbbe-bb126c2d90dd-kube-api-access-hbvtg" (OuterVolumeSpecName: "kube-api-access-hbvtg") pod "f7c0372d-987d-4711-bbbe-bb126c2d90dd" (UID: "f7c0372d-987d-4711-bbbe-bb126c2d90dd"). InnerVolumeSpecName "kube-api-access-hbvtg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:58:25.252756 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.252730 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-util" (OuterVolumeSpecName: "util") pod "f7c0372d-987d-4711-bbbe-bb126c2d90dd" (UID: "f7c0372d-987d-4711-bbbe-bb126c2d90dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:58:25.265508 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.265487 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:25.346625 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.346591 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnhqr\" (UniqueName: \"kubernetes.io/projected/2ac72622-74c6-43f0-8796-700350e39b7a-kube-api-access-fnhqr\") pod \"2ac72622-74c6-43f0-8796-700350e39b7a\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " Jun 10 12:58:25.346837 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.346645 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-bundle\") pod \"2ac72622-74c6-43f0-8796-700350e39b7a\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " Jun 10 12:58:25.346837 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.346670 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-util\") pod \"2ac72622-74c6-43f0-8796-700350e39b7a\" (UID: \"2ac72622-74c6-43f0-8796-700350e39b7a\") " Jun 10 12:58:25.346951 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.346918 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:25.346951 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.346937 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbvtg\" (UniqueName: \"kubernetes.io/projected/f7c0372d-987d-4711-bbbe-bb126c2d90dd-kube-api-access-hbvtg\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:25.347063 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.346955 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c0372d-987d-4711-bbbe-bb126c2d90dd-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:25.347285 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.347253 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-bundle" (OuterVolumeSpecName: "bundle") pod "2ac72622-74c6-43f0-8796-700350e39b7a" (UID: "2ac72622-74c6-43f0-8796-700350e39b7a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:58:25.348771 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.348745 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ac72622-74c6-43f0-8796-700350e39b7a-kube-api-access-fnhqr" (OuterVolumeSpecName: "kube-api-access-fnhqr") pod "2ac72622-74c6-43f0-8796-700350e39b7a" (UID: "2ac72622-74c6-43f0-8796-700350e39b7a"). InnerVolumeSpecName "kube-api-access-fnhqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:58:25.351323 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.351302 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-util" (OuterVolumeSpecName: "util") pod "2ac72622-74c6-43f0-8796-700350e39b7a" (UID: "2ac72622-74c6-43f0-8796-700350e39b7a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:58:25.447359 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.447309 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnhqr\" (UniqueName: \"kubernetes.io/projected/2ac72622-74c6-43f0-8796-700350e39b7a-kube-api-access-fnhqr\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:25.447359 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.447357 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:25.447359 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:25.447373 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ac72622-74c6-43f0-8796-700350e39b7a-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 12:58:26.110310 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:26.110282 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" Jun 10 12:58:26.110841 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:26.110314 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb" event={"ID":"f7c0372d-987d-4711-bbbe-bb126c2d90dd","Type":"ContainerDied","Data":"f582974b8bc2a90444b5973e26d1a81ee5183a67771052afe2438b89b9e06fc7"} Jun 10 12:58:26.110841 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:26.110366 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f582974b8bc2a90444b5973e26d1a81ee5183a67771052afe2438b89b9e06fc7" Jun 10 12:58:26.112165 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:26.112144 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" event={"ID":"2ac72622-74c6-43f0-8796-700350e39b7a","Type":"ContainerDied","Data":"3eaee86cd254c3a3f1860a3da26f8341bbc4e6693b9d64c90a2804f5fbbc0bc7"} Jun 10 12:58:26.112165 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:26.112166 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj" Jun 10 12:58:26.112316 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:26.112176 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3eaee86cd254c3a3f1860a3da26f8341bbc4e6693b9d64c90a2804f5fbbc0bc7" Jun 10 12:58:43.126303 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126263 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm"] Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126537 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerName="extract" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126549 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerName="extract" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126559 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b613471e-a5ab-4552-9d9d-387564bd9280" containerName="extract" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126565 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b613471e-a5ab-4552-9d9d-387564bd9280" containerName="extract" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126572 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerName="pull" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126577 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerName="pull" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126585 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b613471e-a5ab-4552-9d9d-387564bd9280" containerName="util" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126590 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b613471e-a5ab-4552-9d9d-387564bd9280" containerName="util" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126597 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac72622-74c6-43f0-8796-700350e39b7a" containerName="extract" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126602 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac72622-74c6-43f0-8796-700350e39b7a" containerName="extract" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126608 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerName="util" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126613 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerName="util" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126618 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac72622-74c6-43f0-8796-700350e39b7a" containerName="pull" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126622 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac72622-74c6-43f0-8796-700350e39b7a" containerName="pull" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126632 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac72622-74c6-43f0-8796-700350e39b7a" containerName="util" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126636 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac72622-74c6-43f0-8796-700350e39b7a" containerName="util" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126642 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b613471e-a5ab-4552-9d9d-387564bd9280" containerName="pull" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126648 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b613471e-a5ab-4552-9d9d-387564bd9280" containerName="pull" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126655 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerName="extract" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126660 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerName="extract" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126669 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerName="util" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126673 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerName="util" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126679 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerName="pull" Jun 10 12:58:43.126679 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126684 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerName="pull" Jun 10 12:58:43.127397 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126727 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b613471e-a5ab-4552-9d9d-387564bd9280" containerName="extract" Jun 10 12:58:43.127397 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126733 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c0372d-987d-4711-bbbe-bb126c2d90dd" containerName="extract" Jun 10 12:58:43.127397 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126738 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f07091df-a21f-4e5f-bb49-6e9ed46c25c1" containerName="extract" Jun 10 12:58:43.127397 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.126744 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac72622-74c6-43f0-8796-700350e39b7a" containerName="extract" Jun 10 12:58:43.131363 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.131346 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.133749 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.133730 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 10 12:58:43.134015 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.134000 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 10 12:58:43.134762 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.134746 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 10 12:58:43.134871 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.134771 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7xcnb\"" Jun 10 12:58:43.134871 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.134752 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 10 12:58:43.137752 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.137729 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm"] Jun 10 12:58:43.190107 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.190070 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2gzs\" (UniqueName: \"kubernetes.io/projected/51b83f0f-073b-4157-a5b2-0988a2e0213e-kube-api-access-h2gzs\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.190107 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.190114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51b83f0f-073b-4157-a5b2-0988a2e0213e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.190340 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.190261 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/51b83f0f-073b-4157-a5b2-0988a2e0213e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.291215 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.291184 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51b83f0f-073b-4157-a5b2-0988a2e0213e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.291426 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.291264 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/51b83f0f-073b-4157-a5b2-0988a2e0213e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.291426 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.291321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2gzs\" (UniqueName: \"kubernetes.io/projected/51b83f0f-073b-4157-a5b2-0988a2e0213e-kube-api-access-h2gzs\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.291937 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.291910 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51b83f0f-073b-4157-a5b2-0988a2e0213e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.293750 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.293722 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/51b83f0f-073b-4157-a5b2-0988a2e0213e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.299278 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.299256 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2gzs\" (UniqueName: \"kubernetes.io/projected/51b83f0f-073b-4157-a5b2-0988a2e0213e-kube-api-access-h2gzs\") pod \"kuadrant-console-plugin-66b5774985-6hppm\" (UID: \"51b83f0f-073b-4157-a5b2-0988a2e0213e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.442240 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.442139 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" Jun 10 12:58:43.573458 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:43.573433 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm"] Jun 10 12:58:43.575117 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:58:43.575088 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51b83f0f_073b_4157_a5b2_0988a2e0213e.slice/crio-5497ae02ef979db62ba03298476a5b8e4871c1450250aaae5e1eeefa28b1894e WatchSource:0}: Error finding container 5497ae02ef979db62ba03298476a5b8e4871c1450250aaae5e1eeefa28b1894e: Status 404 returned error can't find the container with id 5497ae02ef979db62ba03298476a5b8e4871c1450250aaae5e1eeefa28b1894e Jun 10 12:58:44.204066 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:58:44.204030 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" event={"ID":"51b83f0f-073b-4157-a5b2-0988a2e0213e","Type":"ContainerStarted","Data":"5497ae02ef979db62ba03298476a5b8e4871c1450250aaae5e1eeefa28b1894e"} Jun 10 12:59:09.337869 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:09.337830 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" event={"ID":"51b83f0f-073b-4157-a5b2-0988a2e0213e","Type":"ContainerStarted","Data":"f56f6a296d088e142218b172265983040d2be09bdee87ddff8b13387b68299fb"} Jun 10 12:59:33.516754 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.516687 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6hppm" podStartSLOduration=25.761689332 podStartE2EDuration="50.516663907s" totalImagesPullingTime="24.754974575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:59:09.354709648 +0000 UTC m=+665.061013838" watchObservedRunningTime="2026-06-10 12:59:33.516663907 +0000 UTC m=+689.222968095" Jun 10 12:59:33.518095 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.518074 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 12:59:33.562472 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.562438 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 12:59:33.562472 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.562475 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 12:59:33.562676 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.562557 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:33.565120 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.565087 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 10 12:59:33.601343 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.601313 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46022d38-d4a0-471f-9630-7ad1096ef21f-config-file\") pod \"limitador-limitador-54d6d988d5-spdfr\" (UID: \"46022d38-d4a0-471f-9630-7ad1096ef21f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:33.601496 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.601373 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhzjw\" (UniqueName: \"kubernetes.io/projected/46022d38-d4a0-471f-9630-7ad1096ef21f-kube-api-access-qhzjw\") pod \"limitador-limitador-54d6d988d5-spdfr\" (UID: \"46022d38-d4a0-471f-9630-7ad1096ef21f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:33.701718 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.701684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhzjw\" (UniqueName: \"kubernetes.io/projected/46022d38-d4a0-471f-9630-7ad1096ef21f-kube-api-access-qhzjw\") pod \"limitador-limitador-54d6d988d5-spdfr\" (UID: \"46022d38-d4a0-471f-9630-7ad1096ef21f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:33.701915 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.701741 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46022d38-d4a0-471f-9630-7ad1096ef21f-config-file\") pod \"limitador-limitador-54d6d988d5-spdfr\" (UID: \"46022d38-d4a0-471f-9630-7ad1096ef21f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:33.702370 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.702344 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46022d38-d4a0-471f-9630-7ad1096ef21f-config-file\") pod \"limitador-limitador-54d6d988d5-spdfr\" (UID: \"46022d38-d4a0-471f-9630-7ad1096ef21f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:33.711239 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.711216 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhzjw\" (UniqueName: \"kubernetes.io/projected/46022d38-d4a0-471f-9630-7ad1096ef21f-kube-api-access-qhzjw\") pod \"limitador-limitador-54d6d988d5-spdfr\" (UID: \"46022d38-d4a0-471f-9630-7ad1096ef21f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:33.872269 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:33.872229 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:34.038617 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.038582 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 12:59:34.040035 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:59:34.039998 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46022d38_d4a0_471f_9630_7ad1096ef21f.slice/crio-761a6134fe6f20874062fdf0c169391d2a687c435dc9bec9f14e82807dd6b966 WatchSource:0}: Error finding container 761a6134fe6f20874062fdf0c169391d2a687c435dc9bec9f14e82807dd6b966: Status 404 returned error can't find the container with id 761a6134fe6f20874062fdf0c169391d2a687c435dc9bec9f14e82807dd6b966 Jun 10 12:59:34.186988 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.186905 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-66xlx"] Jun 10 12:59:34.269699 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.269658 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66xlx"] Jun 10 12:59:34.269924 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.269776 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-66xlx" Jun 10 12:59:34.272448 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.272428 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5rld7\"" Jun 10 12:59:34.306791 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.306757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lr5b\" (UniqueName: \"kubernetes.io/projected/2fb7c4f2-7c79-4629-bc54-142d1dc5626f-kube-api-access-5lr5b\") pod \"authorino-75d556bd77-66xlx\" (UID: \"2fb7c4f2-7c79-4629-bc54-142d1dc5626f\") " pod="kuadrant-system/authorino-75d556bd77-66xlx" Jun 10 12:59:34.407778 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.407740 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lr5b\" (UniqueName: \"kubernetes.io/projected/2fb7c4f2-7c79-4629-bc54-142d1dc5626f-kube-api-access-5lr5b\") pod \"authorino-75d556bd77-66xlx\" (UID: \"2fb7c4f2-7c79-4629-bc54-142d1dc5626f\") " pod="kuadrant-system/authorino-75d556bd77-66xlx" Jun 10 12:59:34.416433 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.416406 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lr5b\" (UniqueName: \"kubernetes.io/projected/2fb7c4f2-7c79-4629-bc54-142d1dc5626f-kube-api-access-5lr5b\") pod \"authorino-75d556bd77-66xlx\" (UID: \"2fb7c4f2-7c79-4629-bc54-142d1dc5626f\") " pod="kuadrant-system/authorino-75d556bd77-66xlx" Jun 10 12:59:34.458528 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.458453 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" event={"ID":"46022d38-d4a0-471f-9630-7ad1096ef21f","Type":"ContainerStarted","Data":"761a6134fe6f20874062fdf0c169391d2a687c435dc9bec9f14e82807dd6b966"} Jun 10 12:59:34.579261 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.579215 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-66xlx" Jun 10 12:59:34.759494 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:34.759361 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66xlx"] Jun 10 12:59:34.767589 ip-10-0-140-124 kubenswrapper[2577]: W0610 12:59:34.767548 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fb7c4f2_7c79_4629_bc54_142d1dc5626f.slice/crio-b445998fa0781ec1ca00bd32f5f2cea0d2bd4bd3cb12433e336cba61416a8c95 WatchSource:0}: Error finding container b445998fa0781ec1ca00bd32f5f2cea0d2bd4bd3cb12433e336cba61416a8c95: Status 404 returned error can't find the container with id b445998fa0781ec1ca00bd32f5f2cea0d2bd4bd3cb12433e336cba61416a8c95 Jun 10 12:59:35.466893 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:35.466841 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-66xlx" event={"ID":"2fb7c4f2-7c79-4629-bc54-142d1dc5626f","Type":"ContainerStarted","Data":"b445998fa0781ec1ca00bd32f5f2cea0d2bd4bd3cb12433e336cba61416a8c95"} Jun 10 12:59:39.491901 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:39.491849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-66xlx" event={"ID":"2fb7c4f2-7c79-4629-bc54-142d1dc5626f","Type":"ContainerStarted","Data":"43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81"} Jun 10 12:59:39.493403 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:39.493381 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" event={"ID":"46022d38-d4a0-471f-9630-7ad1096ef21f","Type":"ContainerStarted","Data":"eb52c8938508a86bf856495a061240f475910f784f8df7b72f80f10a6d9a2d97"} Jun 10 12:59:39.493542 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:39.493523 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 12:59:39.511767 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:39.511718 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-66xlx" podStartSLOduration=1.325218418 podStartE2EDuration="5.511700507s" totalImagesPullingTime="4.186482089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:59:39.509111659 +0000 UTC m=+695.215415845" watchObservedRunningTime="2026-06-10 12:59:39.511700507 +0000 UTC m=+695.218004701" Jun 10 12:59:39.534501 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:39.534442 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" podStartSLOduration=1.613130135 podStartE2EDuration="6.534426871s" totalImagesPullingTime="4.921296736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:59:39.53312944 +0000 UTC m=+695.239433632" watchObservedRunningTime="2026-06-10 12:59:39.534426871 +0000 UTC m=+695.240731062" Jun 10 12:59:50.501295 ip-10-0-140-124 kubenswrapper[2577]: I0610 12:59:50.501221 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-spdfr" Jun 10 13:00:06.258924 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.258884 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml"] Jun 10 13:00:06.262307 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.262290 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.265037 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.265018 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 13:00:06.266082 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.266060 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 13:00:06.266173 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.266092 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jcl4f\"" Jun 10 13:00:06.271672 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.271639 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml"] Jun 10 13:00:06.399644 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.399608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stzp8\" (UniqueName: \"kubernetes.io/projected/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-kube-api-access-stzp8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.399847 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.399662 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.399847 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.399701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.500474 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.500441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stzp8\" (UniqueName: \"kubernetes.io/projected/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-kube-api-access-stzp8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.500602 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.500488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.500644 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.500607 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.500976 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.500961 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.501018 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.501004 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.510245 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.510182 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stzp8\" (UniqueName: \"kubernetes.io/projected/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-kube-api-access-stzp8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.572241 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.572192 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:06.703987 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:06.703959 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml"] Jun 10 13:00:06.705243 ip-10-0-140-124 kubenswrapper[2577]: W0610 13:00:06.705220 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0354f2_b766_4ba5_b0a1_71d4b878ed0a.slice/crio-ba9ea810a751980055fbd96a27865dce553f42bac4ce5bf367c2d012980f9a4c WatchSource:0}: Error finding container ba9ea810a751980055fbd96a27865dce553f42bac4ce5bf367c2d012980f9a4c: Status 404 returned error can't find the container with id ba9ea810a751980055fbd96a27865dce553f42bac4ce5bf367c2d012980f9a4c Jun 10 13:00:07.630584 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:07.630542 2577 generic.go:356] "Generic (PLEG): container finished" podID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerID="6c26071a8b2edbdf2f9e2dac376b99171b3d275a77b0f92712023a6f4bba7d5b" exitCode=0 Jun 10 13:00:07.631000 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:07.630598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" event={"ID":"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a","Type":"ContainerDied","Data":"6c26071a8b2edbdf2f9e2dac376b99171b3d275a77b0f92712023a6f4bba7d5b"} Jun 10 13:00:07.631000 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:07.630645 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" event={"ID":"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a","Type":"ContainerStarted","Data":"ba9ea810a751980055fbd96a27865dce553f42bac4ce5bf367c2d012980f9a4c"} Jun 10 13:00:08.637306 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:08.637269 2577 generic.go:356] "Generic (PLEG): container finished" podID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerID="02a27321665fba498e098e9e263e8a11a31929520b7d4f4b13d5e455e37aa7cc" exitCode=0 Jun 10 13:00:08.637729 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:08.637358 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" event={"ID":"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a","Type":"ContainerDied","Data":"02a27321665fba498e098e9e263e8a11a31929520b7d4f4b13d5e455e37aa7cc"} Jun 10 13:00:09.644649 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:09.644611 2577 generic.go:356] "Generic (PLEG): container finished" podID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerID="9978d090dac349f686bd8e88764042914267a857ea86e46cefa5f5303fe13378" exitCode=0 Jun 10 13:00:09.645046 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:09.644660 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" event={"ID":"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a","Type":"ContainerDied","Data":"9978d090dac349f686bd8e88764042914267a857ea86e46cefa5f5303fe13378"} Jun 10 13:00:10.779162 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:10.779128 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:10.938611 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:10.938520 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stzp8\" (UniqueName: \"kubernetes.io/projected/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-kube-api-access-stzp8\") pod \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " Jun 10 13:00:10.938611 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:10.938594 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-util\") pod \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " Jun 10 13:00:10.938846 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:10.938615 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-bundle\") pod \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\" (UID: \"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a\") " Jun 10 13:00:10.939206 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:10.939174 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-bundle" (OuterVolumeSpecName: "bundle") pod "cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" (UID: "cc0354f2-b766-4ba5-b0a1-71d4b878ed0a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:00:10.940691 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:10.940666 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-kube-api-access-stzp8" (OuterVolumeSpecName: "kube-api-access-stzp8") pod "cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" (UID: "cc0354f2-b766-4ba5-b0a1-71d4b878ed0a"). InnerVolumeSpecName "kube-api-access-stzp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:00:10.944703 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:10.944681 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-util" (OuterVolumeSpecName: "util") pod "cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" (UID: "cc0354f2-b766-4ba5-b0a1-71d4b878ed0a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:00:11.039758 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:11.039720 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stzp8\" (UniqueName: \"kubernetes.io/projected/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-kube-api-access-stzp8\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:00:11.039758 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:11.039752 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-util\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:00:11.039758 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:11.039762 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc0354f2-b766-4ba5-b0a1-71d4b878ed0a-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:00:11.657291 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:11.657257 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" event={"ID":"cc0354f2-b766-4ba5-b0a1-71d4b878ed0a","Type":"ContainerDied","Data":"ba9ea810a751980055fbd96a27865dce553f42bac4ce5bf367c2d012980f9a4c"} Jun 10 13:00:11.657291 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:11.657278 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748vf4ml" Jun 10 13:00:11.657569 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:00:11.657305 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba9ea810a751980055fbd96a27865dce553f42bac4ce5bf367c2d012980f9a4c" Jun 10 13:01:00.170774 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.170735 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v8vpj"] Jun 10 13:01:00.171217 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.171044 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerName="extract" Jun 10 13:01:00.171217 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.171058 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerName="extract" Jun 10 13:01:00.171217 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.171081 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerName="pull" Jun 10 13:01:00.171217 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.171087 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerName="pull" Jun 10 13:01:00.171217 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.171094 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerName="util" Jun 10 13:01:00.171217 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.171100 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerName="util" Jun 10 13:01:00.171217 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.171146 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0354f2-b766-4ba5-b0a1-71d4b878ed0a" containerName="extract" Jun 10 13:01:00.173985 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.173968 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-v8vpj" Jun 10 13:01:00.181516 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.181497 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v8vpj"] Jun 10 13:01:00.351160 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.351121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2trpk\" (UniqueName: \"kubernetes.io/projected/21bf212c-929f-4e92-ad13-83b761526430-kube-api-access-2trpk\") pod \"authorino-5d4b67c446-v8vpj\" (UID: \"21bf212c-929f-4e92-ad13-83b761526430\") " pod="kuadrant-system/authorino-5d4b67c446-v8vpj" Jun 10 13:01:00.407505 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.407464 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v8vpj"] Jun 10 13:01:00.407762 ip-10-0-140-124 kubenswrapper[2577]: E0610 13:01:00.407738 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2trpk], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-v8vpj" podUID="21bf212c-929f-4e92-ad13-83b761526430" Jun 10 13:01:00.437951 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.437877 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-599bd84cdf-g6dc8"] Jun 10 13:01:00.441029 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.441012 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-599bd84cdf-g6dc8" Jun 10 13:01:00.452136 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.452116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2trpk\" (UniqueName: \"kubernetes.io/projected/21bf212c-929f-4e92-ad13-83b761526430-kube-api-access-2trpk\") pod \"authorino-5d4b67c446-v8vpj\" (UID: \"21bf212c-929f-4e92-ad13-83b761526430\") " pod="kuadrant-system/authorino-5d4b67c446-v8vpj" Jun 10 13:01:00.454775 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.454753 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-599bd84cdf-g6dc8"] Jun 10 13:01:00.464130 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.464110 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2trpk\" (UniqueName: \"kubernetes.io/projected/21bf212c-929f-4e92-ad13-83b761526430-kube-api-access-2trpk\") pod \"authorino-5d4b67c446-v8vpj\" (UID: \"21bf212c-929f-4e92-ad13-83b761526430\") " pod="kuadrant-system/authorino-5d4b67c446-v8vpj" Jun 10 13:01:00.553405 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.553361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvztk\" (UniqueName: \"kubernetes.io/projected/b1fc138a-6460-4f9a-b4db-23f1816f398c-kube-api-access-hvztk\") pod \"authorino-599bd84cdf-g6dc8\" (UID: \"b1fc138a-6460-4f9a-b4db-23f1816f398c\") " pod="kuadrant-system/authorino-599bd84cdf-g6dc8" Jun 10 13:01:00.628152 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.628117 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-599bd84cdf-g6dc8"] Jun 10 13:01:00.628368 ip-10-0-140-124 kubenswrapper[2577]: E0610 13:01:00.628351 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hvztk], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-599bd84cdf-g6dc8" podUID="b1fc138a-6460-4f9a-b4db-23f1816f398c" Jun 10 13:01:00.654474 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.654442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvztk\" (UniqueName: \"kubernetes.io/projected/b1fc138a-6460-4f9a-b4db-23f1816f398c-kube-api-access-hvztk\") pod \"authorino-599bd84cdf-g6dc8\" (UID: \"b1fc138a-6460-4f9a-b4db-23f1816f398c\") " pod="kuadrant-system/authorino-599bd84cdf-g6dc8" Jun 10 13:01:00.669549 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.669519 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvztk\" (UniqueName: \"kubernetes.io/projected/b1fc138a-6460-4f9a-b4db-23f1816f398c-kube-api-access-hvztk\") pod \"authorino-599bd84cdf-g6dc8\" (UID: \"b1fc138a-6460-4f9a-b4db-23f1816f398c\") " pod="kuadrant-system/authorino-599bd84cdf-g6dc8" Jun 10 13:01:00.896838 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.896789 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-599bd84cdf-g6dc8" Jun 10 13:01:00.897111 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.896789 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-v8vpj" Jun 10 13:01:00.903890 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.903869 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-599bd84cdf-g6dc8" Jun 10 13:01:00.906829 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:00.906791 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-v8vpj" Jun 10 13:01:01.057283 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.057249 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvztk\" (UniqueName: \"kubernetes.io/projected/b1fc138a-6460-4f9a-b4db-23f1816f398c-kube-api-access-hvztk\") pod \"b1fc138a-6460-4f9a-b4db-23f1816f398c\" (UID: \"b1fc138a-6460-4f9a-b4db-23f1816f398c\") " Jun 10 13:01:01.057470 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.057345 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2trpk\" (UniqueName: \"kubernetes.io/projected/21bf212c-929f-4e92-ad13-83b761526430-kube-api-access-2trpk\") pod \"21bf212c-929f-4e92-ad13-83b761526430\" (UID: \"21bf212c-929f-4e92-ad13-83b761526430\") " Jun 10 13:01:01.059368 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.059342 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21bf212c-929f-4e92-ad13-83b761526430-kube-api-access-2trpk" (OuterVolumeSpecName: "kube-api-access-2trpk") pod "21bf212c-929f-4e92-ad13-83b761526430" (UID: "21bf212c-929f-4e92-ad13-83b761526430"). InnerVolumeSpecName "kube-api-access-2trpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:01:01.059454 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.059425 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1fc138a-6460-4f9a-b4db-23f1816f398c-kube-api-access-hvztk" (OuterVolumeSpecName: "kube-api-access-hvztk") pod "b1fc138a-6460-4f9a-b4db-23f1816f398c" (UID: "b1fc138a-6460-4f9a-b4db-23f1816f398c"). InnerVolumeSpecName "kube-api-access-hvztk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:01:01.158125 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.158037 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2trpk\" (UniqueName: \"kubernetes.io/projected/21bf212c-929f-4e92-ad13-83b761526430-kube-api-access-2trpk\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:01:01.158125 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.158068 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvztk\" (UniqueName: \"kubernetes.io/projected/b1fc138a-6460-4f9a-b4db-23f1816f398c-kube-api-access-hvztk\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:01:01.901440 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.901408 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-v8vpj" Jun 10 13:01:01.901868 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.901409 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-599bd84cdf-g6dc8" Jun 10 13:01:01.959124 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.959090 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v8vpj"] Jun 10 13:01:01.962602 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.962577 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v8vpj"] Jun 10 13:01:01.997115 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.997078 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-599bd84cdf-g6dc8"] Jun 10 13:01:01.999474 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:01.999449 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-599bd84cdf-g6dc8"] Jun 10 13:01:02.791066 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:02.791036 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21bf212c-929f-4e92-ad13-83b761526430" path="/var/lib/kubelet/pods/21bf212c-929f-4e92-ad13-83b761526430/volumes" Jun 10 13:01:02.791257 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:02.791246 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1fc138a-6460-4f9a-b4db-23f1816f398c" path="/var/lib/kubelet/pods/b1fc138a-6460-4f9a-b4db-23f1816f398c/volumes" Jun 10 13:01:03.044742 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.044660 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66xlx"] Jun 10 13:01:03.045182 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.044850 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-66xlx" podUID="2fb7c4f2-7c79-4629-bc54-142d1dc5626f" containerName="authorino" containerID="cri-o://43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81" gracePeriod=30 Jun 10 13:01:03.289815 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.289778 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-66xlx" Jun 10 13:01:03.374321 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.374291 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lr5b\" (UniqueName: \"kubernetes.io/projected/2fb7c4f2-7c79-4629-bc54-142d1dc5626f-kube-api-access-5lr5b\") pod \"2fb7c4f2-7c79-4629-bc54-142d1dc5626f\" (UID: \"2fb7c4f2-7c79-4629-bc54-142d1dc5626f\") " Jun 10 13:01:03.376395 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.376367 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fb7c4f2-7c79-4629-bc54-142d1dc5626f-kube-api-access-5lr5b" (OuterVolumeSpecName: "kube-api-access-5lr5b") pod "2fb7c4f2-7c79-4629-bc54-142d1dc5626f" (UID: "2fb7c4f2-7c79-4629-bc54-142d1dc5626f"). InnerVolumeSpecName "kube-api-access-5lr5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:01:03.475428 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.475393 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lr5b\" (UniqueName: \"kubernetes.io/projected/2fb7c4f2-7c79-4629-bc54-142d1dc5626f-kube-api-access-5lr5b\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:01:03.912082 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.912050 2577 generic.go:356] "Generic (PLEG): container finished" podID="2fb7c4f2-7c79-4629-bc54-142d1dc5626f" containerID="43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81" exitCode=0 Jun 10 13:01:03.912264 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.912095 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-66xlx" Jun 10 13:01:03.912264 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.912138 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-66xlx" event={"ID":"2fb7c4f2-7c79-4629-bc54-142d1dc5626f","Type":"ContainerDied","Data":"43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81"} Jun 10 13:01:03.912264 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.912182 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-66xlx" event={"ID":"2fb7c4f2-7c79-4629-bc54-142d1dc5626f","Type":"ContainerDied","Data":"b445998fa0781ec1ca00bd32f5f2cea0d2bd4bd3cb12433e336cba61416a8c95"} Jun 10 13:01:03.912264 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.912204 2577 scope.go:117] "RemoveContainer" containerID="43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81" Jun 10 13:01:03.920951 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.920933 2577 scope.go:117] "RemoveContainer" containerID="43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81" Jun 10 13:01:03.921213 ip-10-0-140-124 kubenswrapper[2577]: E0610 13:01:03.921194 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81\": container with ID starting with 43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81 not found: ID does not exist" containerID="43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81" Jun 10 13:01:03.921275 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.921225 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81"} err="failed to get container status \"43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81\": rpc error: code = NotFound desc = could not find container \"43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81\": container with ID starting with 43c0e09f26baecb5a770fc121ba4abdd4840efffcc5661e4f14ecc2a29f6bd81 not found: ID does not exist" Jun 10 13:01:03.932664 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.932640 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66xlx"] Jun 10 13:01:03.935873 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:03.935853 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66xlx"] Jun 10 13:01:04.793598 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:04.793568 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fb7c4f2-7c79-4629-bc54-142d1dc5626f" path="/var/lib/kubelet/pods/2fb7c4f2-7c79-4629-bc54-142d1dc5626f/volumes" Jun 10 13:01:41.078479 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.078388 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm"] Jun 10 13:01:41.079059 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.078842 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fb7c4f2-7c79-4629-bc54-142d1dc5626f" containerName="authorino" Jun 10 13:01:41.079059 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.078860 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fb7c4f2-7c79-4629-bc54-142d1dc5626f" containerName="authorino" Jun 10 13:01:41.079059 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.078938 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fb7c4f2-7c79-4629-bc54-142d1dc5626f" containerName="authorino" Jun 10 13:01:41.081794 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.081775 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:41.084488 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.084460 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-dockercfg-bjdnz\"" Jun 10 13:01:41.084604 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.084535 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jun 10 13:01:41.085497 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.085480 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jun 10 13:01:41.085616 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.085522 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jun 10 13:01:41.097907 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.097554 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm"] Jun 10 13:01:41.171082 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.171039 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jltl\" (UniqueName: \"kubernetes.io/projected/01199f77-1ccb-49b4-b5d4-358f66682993-kube-api-access-8jltl\") pod \"maas-api-7dd4bdcf5c-rh8pm\" (UID: \"01199f77-1ccb-49b4-b5d4-358f66682993\") " pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:41.171256 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.171093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01199f77-1ccb-49b4-b5d4-358f66682993-maas-api-tls\") pod \"maas-api-7dd4bdcf5c-rh8pm\" (UID: \"01199f77-1ccb-49b4-b5d4-358f66682993\") " pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:41.272251 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.272215 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jltl\" (UniqueName: \"kubernetes.io/projected/01199f77-1ccb-49b4-b5d4-358f66682993-kube-api-access-8jltl\") pod \"maas-api-7dd4bdcf5c-rh8pm\" (UID: \"01199f77-1ccb-49b4-b5d4-358f66682993\") " pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:41.272251 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.272257 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01199f77-1ccb-49b4-b5d4-358f66682993-maas-api-tls\") pod \"maas-api-7dd4bdcf5c-rh8pm\" (UID: \"01199f77-1ccb-49b4-b5d4-358f66682993\") " pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:41.272466 ip-10-0-140-124 kubenswrapper[2577]: E0610 13:01:41.272351 2577 secret.go:189] Couldn't get secret redhat-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 10 13:01:41.272466 ip-10-0-140-124 kubenswrapper[2577]: E0610 13:01:41.272434 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01199f77-1ccb-49b4-b5d4-358f66682993-maas-api-tls podName:01199f77-1ccb-49b4-b5d4-358f66682993 nodeName:}" failed. No retries permitted until 2026-06-10 13:01:41.772417436 +0000 UTC m=+817.478721604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/01199f77-1ccb-49b4-b5d4-358f66682993-maas-api-tls") pod "maas-api-7dd4bdcf5c-rh8pm" (UID: "01199f77-1ccb-49b4-b5d4-358f66682993") : secret "maas-api-serving-cert" not found Jun 10 13:01:41.282390 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.282366 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jltl\" (UniqueName: \"kubernetes.io/projected/01199f77-1ccb-49b4-b5d4-358f66682993-kube-api-access-8jltl\") pod \"maas-api-7dd4bdcf5c-rh8pm\" (UID: \"01199f77-1ccb-49b4-b5d4-358f66682993\") " pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:41.776228 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.776194 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01199f77-1ccb-49b4-b5d4-358f66682993-maas-api-tls\") pod \"maas-api-7dd4bdcf5c-rh8pm\" (UID: \"01199f77-1ccb-49b4-b5d4-358f66682993\") " pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:41.778569 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.778547 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01199f77-1ccb-49b4-b5d4-358f66682993-maas-api-tls\") pod \"maas-api-7dd4bdcf5c-rh8pm\" (UID: \"01199f77-1ccb-49b4-b5d4-358f66682993\") " pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:41.782610 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.782581 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-dgmbc"] Jun 10 13:01:41.785899 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.785880 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" Jun 10 13:01:41.788318 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.788297 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xqnpq\"" Jun 10 13:01:41.788415 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.788328 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 10 13:01:41.795709 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.795686 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-dgmbc"] Jun 10 13:01:41.876906 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.876867 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5485\" (UniqueName: \"kubernetes.io/projected/b724625f-9d67-4649-bc9e-73da596fdde1-kube-api-access-z5485\") pod \"payload-pre-processing-579785bc64-dgmbc\" (UID: \"b724625f-9d67-4649-bc9e-73da596fdde1\") " pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" Jun 10 13:01:41.914026 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.913999 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:01:41.978066 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.978033 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5485\" (UniqueName: \"kubernetes.io/projected/b724625f-9d67-4649-bc9e-73da596fdde1-kube-api-access-z5485\") pod \"payload-pre-processing-579785bc64-dgmbc\" (UID: \"b724625f-9d67-4649-bc9e-73da596fdde1\") " pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" Jun 10 13:01:41.986298 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.986272 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5485\" (UniqueName: \"kubernetes.io/projected/b724625f-9d67-4649-bc9e-73da596fdde1-kube-api-access-z5485\") pod \"payload-pre-processing-579785bc64-dgmbc\" (UID: \"b724625f-9d67-4649-bc9e-73da596fdde1\") " pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" Jun 10 13:01:41.992056 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:41.992034 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:42.096356 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:42.096307 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" Jun 10 13:01:42.122627 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:42.122600 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm"] Jun 10 13:01:42.124268 ip-10-0-140-124 kubenswrapper[2577]: W0610 13:01:42.124242 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01199f77_1ccb_49b4_b5d4_358f66682993.slice/crio-8a28a7b395eecfdf9806133e1e7777bec74f069982e956bf19d5f8e1799601df WatchSource:0}: Error finding container 8a28a7b395eecfdf9806133e1e7777bec74f069982e956bf19d5f8e1799601df: Status 404 returned error can't find the container with id 8a28a7b395eecfdf9806133e1e7777bec74f069982e956bf19d5f8e1799601df Jun 10 13:01:42.224503 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:42.224482 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-dgmbc"] Jun 10 13:01:42.226011 ip-10-0-140-124 kubenswrapper[2577]: W0610 13:01:42.225978 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb724625f_9d67_4649_bc9e_73da596fdde1.slice/crio-065f167554eb483c35519cb0d131b562c7eff1716896187246b08d49a6de01a7 WatchSource:0}: Error finding container 065f167554eb483c35519cb0d131b562c7eff1716896187246b08d49a6de01a7: Status 404 returned error can't find the container with id 065f167554eb483c35519cb0d131b562c7eff1716896187246b08d49a6de01a7 Jun 10 13:01:43.107355 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:43.107315 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" event={"ID":"01199f77-1ccb-49b4-b5d4-358f66682993","Type":"ContainerStarted","Data":"8a28a7b395eecfdf9806133e1e7777bec74f069982e956bf19d5f8e1799601df"} Jun 10 13:01:43.109884 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:43.109850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" event={"ID":"b724625f-9d67-4649-bc9e-73da596fdde1","Type":"ContainerStarted","Data":"065f167554eb483c35519cb0d131b562c7eff1716896187246b08d49a6de01a7"} Jun 10 13:01:46.127430 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:46.127398 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" event={"ID":"01199f77-1ccb-49b4-b5d4-358f66682993","Type":"ContainerStarted","Data":"6a478310b65c40a62e5915629bc4c08ddeef6cd80e0f6639cef2b1e830768206"} Jun 10 13:01:46.127932 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:46.127493 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:46.128964 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:46.128941 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" event={"ID":"b724625f-9d67-4649-bc9e-73da596fdde1","Type":"ContainerStarted","Data":"35698ab9432d8dd4c6815bd0ca944134dcf9e98900425b49666013ca795696ea"} Jun 10 13:01:46.129094 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:46.129074 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" Jun 10 13:01:46.145060 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:46.145009 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" podStartSLOduration=1.829449875 podStartE2EDuration="5.144990998s" totalImagesPullingTime="3.315541123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:01:46.144448547 +0000 UTC m=+821.850752736" watchObservedRunningTime="2026-06-10 13:01:46.144990998 +0000 UTC m=+821.851295188" Jun 10 13:01:46.171072 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:46.171022 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" podStartSLOduration=1.909492027 podStartE2EDuration="5.171007952s" totalImagesPullingTime="3.261515925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:01:46.168412541 +0000 UTC m=+821.874716732" watchObservedRunningTime="2026-06-10 13:01:46.171007952 +0000 UTC m=+821.877312141" Jun 10 13:01:52.140390 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:52.140360 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/maas-api-7dd4bdcf5c-rh8pm" Jun 10 13:01:57.137136 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:01:57.137103 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-dgmbc" Jun 10 13:02:41.770207 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.770166 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7545cf9777-4pftk"] Jun 10 13:02:41.773339 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.773323 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.775974 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.775951 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 10 13:02:41.776966 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.776951 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5rld7\"" Jun 10 13:02:41.777030 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.776987 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 10 13:02:41.786209 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.786181 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7545cf9777-4pftk"] Jun 10 13:02:41.876747 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.876711 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-oidc-ca\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.876933 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.876772 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-tls-cert\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.876933 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.876851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwqn5\" (UniqueName: \"kubernetes.io/projected/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-kube-api-access-mwqn5\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.977903 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.977858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqn5\" (UniqueName: \"kubernetes.io/projected/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-kube-api-access-mwqn5\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.978081 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.977939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-oidc-ca\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.978081 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.977988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-tls-cert\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.978683 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.978657 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-oidc-ca\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.980157 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.980131 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-tls-cert\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:41.985696 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:41.985676 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwqn5\" (UniqueName: \"kubernetes.io/projected/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-kube-api-access-mwqn5\") pod \"authorino-7545cf9777-4pftk\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:42.084668 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:42.084638 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:02:42.216628 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:42.216605 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7545cf9777-4pftk"] Jun 10 13:02:42.218756 ip-10-0-140-124 kubenswrapper[2577]: W0610 13:02:42.218725 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49fa1aa7_ec20_45f5_a9ca_a3bf40f92ad9.slice/crio-9b0a93cd0682485e0a9d1525251542022020d0e8910d8450e677494d4489bccc WatchSource:0}: Error finding container 9b0a93cd0682485e0a9d1525251542022020d0e8910d8450e677494d4489bccc: Status 404 returned error can't find the container with id 9b0a93cd0682485e0a9d1525251542022020d0e8910d8450e677494d4489bccc Jun 10 13:02:42.400949 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:42.400913 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545cf9777-4pftk" event={"ID":"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9","Type":"ContainerStarted","Data":"9b0a93cd0682485e0a9d1525251542022020d0e8910d8450e677494d4489bccc"} Jun 10 13:02:43.407319 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:43.407277 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545cf9777-4pftk" event={"ID":"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9","Type":"ContainerStarted","Data":"7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd"} Jun 10 13:02:43.424021 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:43.423974 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7545cf9777-4pftk" podStartSLOduration=2.035119728 podStartE2EDuration="2.423957066s" totalImagesPullingTime="388.837338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:02:43.422779732 +0000 UTC m=+879.129083917" watchObservedRunningTime="2026-06-10 13:02:43.423957066 +0000 UTC m=+879.130261258" Jun 10 13:02:53.993414 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:02:53.993316 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:03:02.891904 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:02.891867 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:03:04.801521 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:04.801497 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:03:04.803191 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:04.803171 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:03:15.090681 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:15.090644 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:03:21.296179 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:21.296142 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:03:40.094152 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:40.094115 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:03:41.354465 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.354423 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c4786f77f-r9pdp"] Jun 10 13:03:41.357837 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.357794 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.369493 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.369470 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4786f77f-r9pdp"] Jun 10 13:03:41.469794 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.469753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65s58\" (UniqueName: \"kubernetes.io/projected/98c990c9-0e68-41da-8c00-c950429a31c5-kube-api-access-65s58\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.469794 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.469798 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-service-ca\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.470035 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.469838 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98c990c9-0e68-41da-8c00-c950429a31c5-console-serving-cert\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.470035 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.469890 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98c990c9-0e68-41da-8c00-c950429a31c5-console-oauth-config\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.470035 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.469906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-trusted-ca-bundle\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.470035 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.469935 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-console-config\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.470035 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.469959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-oauth-serving-cert\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.571247 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.571209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65s58\" (UniqueName: \"kubernetes.io/projected/98c990c9-0e68-41da-8c00-c950429a31c5-kube-api-access-65s58\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.571454 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.571259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-service-ca\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.571454 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.571292 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98c990c9-0e68-41da-8c00-c950429a31c5-console-serving-cert\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.571454 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.571322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98c990c9-0e68-41da-8c00-c950429a31c5-console-oauth-config\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.571454 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.571343 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-trusted-ca-bundle\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.571454 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.571386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-console-config\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.571454 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.571410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-oauth-serving-cert\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.572203 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.572178 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-service-ca\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.572339 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.572272 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-console-config\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.572420 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.572361 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-oauth-serving-cert\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.572456 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.572434 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98c990c9-0e68-41da-8c00-c950429a31c5-trusted-ca-bundle\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.573969 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.573951 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98c990c9-0e68-41da-8c00-c950429a31c5-console-serving-cert\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.573969 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.573963 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98c990c9-0e68-41da-8c00-c950429a31c5-console-oauth-config\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.579774 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.579754 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65s58\" (UniqueName: \"kubernetes.io/projected/98c990c9-0e68-41da-8c00-c950429a31c5-kube-api-access-65s58\") pod \"console-6c4786f77f-r9pdp\" (UID: \"98c990c9-0e68-41da-8c00-c950429a31c5\") " pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.668144 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.668113 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:41.805762 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:41.805592 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4786f77f-r9pdp"] Jun 10 13:03:41.808447 ip-10-0-140-124 kubenswrapper[2577]: W0610 13:03:41.808414 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c990c9_0e68_41da_8c00_c950429a31c5.slice/crio-971662949c6e721bd672409e4e4c53014551f58dc09e9f75bccaba5e599bca6e WatchSource:0}: Error finding container 971662949c6e721bd672409e4e4c53014551f58dc09e9f75bccaba5e599bca6e: Status 404 returned error can't find the container with id 971662949c6e721bd672409e4e4c53014551f58dc09e9f75bccaba5e599bca6e Jun 10 13:03:42.704120 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:42.704082 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4786f77f-r9pdp" event={"ID":"98c990c9-0e68-41da-8c00-c950429a31c5","Type":"ContainerStarted","Data":"c4578b145c7a619e42e3becd8352e607f4ed751187dd739e69f66f847dc2fc2d"} Jun 10 13:03:42.704564 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:42.704158 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4786f77f-r9pdp" event={"ID":"98c990c9-0e68-41da-8c00-c950429a31c5","Type":"ContainerStarted","Data":"971662949c6e721bd672409e4e4c53014551f58dc09e9f75bccaba5e599bca6e"} Jun 10 13:03:42.723590 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:42.723539 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c4786f77f-r9pdp" podStartSLOduration=1.723523975 podStartE2EDuration="1.723523975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:03:42.721822402 +0000 UTC m=+938.428126584" watchObservedRunningTime="2026-06-10 13:03:42.723523975 +0000 UTC m=+938.429828173" Jun 10 13:03:47.390049 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:47.390016 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:03:51.668995 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:51.668946 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:51.669404 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:51.669017 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:51.673774 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:51.673754 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:51.753305 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:51.753278 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c4786f77f-r9pdp" Jun 10 13:03:51.808418 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:03:51.808385 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86cfc994bd-mjdxc"] Jun 10 13:04:16.832161 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:16.832097 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-86cfc994bd-mjdxc" podUID="6d6f444f-4a19-4308-8858-5552aae7ea15" containerName="console" containerID="cri-o://a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b" gracePeriod=15 Jun 10 13:04:17.078508 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.078486 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 13:04:17.168434 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.168400 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-oauth-serving-cert\") pod \"6d6f444f-4a19-4308-8858-5552aae7ea15\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " Jun 10 13:04:17.168609 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.168443 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxcls\" (UniqueName: \"kubernetes.io/projected/6d6f444f-4a19-4308-8858-5552aae7ea15-kube-api-access-fxcls\") pod \"6d6f444f-4a19-4308-8858-5552aae7ea15\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " Jun 10 13:04:17.168609 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.168479 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-serving-cert\") pod \"6d6f444f-4a19-4308-8858-5552aae7ea15\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " Jun 10 13:04:17.168609 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.168507 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-oauth-config\") pod \"6d6f444f-4a19-4308-8858-5552aae7ea15\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " Jun 10 13:04:17.168609 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.168534 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-trusted-ca-bundle\") pod \"6d6f444f-4a19-4308-8858-5552aae7ea15\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " Jun 10 13:04:17.168609 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.168599 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-console-config\") pod \"6d6f444f-4a19-4308-8858-5552aae7ea15\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " Jun 10 13:04:17.168899 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.168631 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-service-ca\") pod \"6d6f444f-4a19-4308-8858-5552aae7ea15\" (UID: \"6d6f444f-4a19-4308-8858-5552aae7ea15\") " Jun 10 13:04:17.168899 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.168843 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6d6f444f-4a19-4308-8858-5552aae7ea15" (UID: "6d6f444f-4a19-4308-8858-5552aae7ea15"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:04:17.169124 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.169097 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-console-config" (OuterVolumeSpecName: "console-config") pod "6d6f444f-4a19-4308-8858-5552aae7ea15" (UID: "6d6f444f-4a19-4308-8858-5552aae7ea15"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:04:17.169124 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.169105 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6d6f444f-4a19-4308-8858-5552aae7ea15" (UID: "6d6f444f-4a19-4308-8858-5552aae7ea15"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:04:17.169281 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.169159 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-service-ca" (OuterVolumeSpecName: "service-ca") pod "6d6f444f-4a19-4308-8858-5552aae7ea15" (UID: "6d6f444f-4a19-4308-8858-5552aae7ea15"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:04:17.170817 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.170779 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6d6f444f-4a19-4308-8858-5552aae7ea15" (UID: "6d6f444f-4a19-4308-8858-5552aae7ea15"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 13:04:17.171171 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.171150 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6d6f444f-4a19-4308-8858-5552aae7ea15" (UID: "6d6f444f-4a19-4308-8858-5552aae7ea15"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 13:04:17.171171 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.171158 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d6f444f-4a19-4308-8858-5552aae7ea15-kube-api-access-fxcls" (OuterVolumeSpecName: "kube-api-access-fxcls") pod "6d6f444f-4a19-4308-8858-5552aae7ea15" (UID: "6d6f444f-4a19-4308-8858-5552aae7ea15"). InnerVolumeSpecName "kube-api-access-fxcls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:04:17.269887 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.269835 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxcls\" (UniqueName: \"kubernetes.io/projected/6d6f444f-4a19-4308-8858-5552aae7ea15-kube-api-access-fxcls\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:17.269887 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.269880 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-serving-cert\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:17.269887 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.269893 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d6f444f-4a19-4308-8858-5552aae7ea15-console-oauth-config\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:17.270141 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.269908 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-trusted-ca-bundle\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:17.270141 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.269921 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-console-config\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:17.270141 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.269932 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-service-ca\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:17.270141 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.269944 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d6f444f-4a19-4308-8858-5552aae7ea15-oauth-serving-cert\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:17.889190 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.889158 2577 generic.go:356] "Generic (PLEG): container finished" podID="6d6f444f-4a19-4308-8858-5552aae7ea15" containerID="a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b" exitCode=0 Jun 10 13:04:17.889617 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.889198 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cfc994bd-mjdxc" Jun 10 13:04:17.889617 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.889241 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cfc994bd-mjdxc" event={"ID":"6d6f444f-4a19-4308-8858-5552aae7ea15","Type":"ContainerDied","Data":"a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b"} Jun 10 13:04:17.889617 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.889292 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cfc994bd-mjdxc" event={"ID":"6d6f444f-4a19-4308-8858-5552aae7ea15","Type":"ContainerDied","Data":"b6d8ccebcb5f600a115c9a9ad9b47588de948de223c3ac517d574a955da5125b"} Jun 10 13:04:17.889617 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.889311 2577 scope.go:117] "RemoveContainer" containerID="a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b" Jun 10 13:04:17.898339 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.898323 2577 scope.go:117] "RemoveContainer" containerID="a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b" Jun 10 13:04:17.898599 ip-10-0-140-124 kubenswrapper[2577]: E0610 13:04:17.898582 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b\": container with ID starting with a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b not found: ID does not exist" containerID="a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b" Jun 10 13:04:17.898639 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.898607 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b"} err="failed to get container status \"a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b\": rpc error: code = NotFound desc = could not find container \"a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b\": container with ID starting with a8864f7491cd6915f266de0274a5f310a04dc745adb7245a419f9553b1a5180b not found: ID does not exist" Jun 10 13:04:17.911135 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.911109 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86cfc994bd-mjdxc"] Jun 10 13:04:17.915897 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:17.915871 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86cfc994bd-mjdxc"] Jun 10 13:04:18.791329 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:18.791297 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d6f444f-4a19-4308-8858-5552aae7ea15" path="/var/lib/kubelet/pods/6d6f444f-4a19-4308-8858-5552aae7ea15/volumes" Jun 10 13:04:30.342219 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.342179 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f68bd4764-mtjzz"] Jun 10 13:04:30.342629 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.342488 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d6f444f-4a19-4308-8858-5552aae7ea15" containerName="console" Jun 10 13:04:30.342629 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.342500 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d6f444f-4a19-4308-8858-5552aae7ea15" containerName="console" Jun 10 13:04:30.342629 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.342564 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d6f444f-4a19-4308-8858-5552aae7ea15" containerName="console" Jun 10 13:04:30.345388 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.345372 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.355173 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.355146 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f68bd4764-mtjzz"] Jun 10 13:04:30.475353 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.475302 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-tls-cert\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.475353 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.475357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-oidc-ca\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.475575 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.475385 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mr5v\" (UniqueName: \"kubernetes.io/projected/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-kube-api-access-9mr5v\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.576456 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.576424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-tls-cert\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.576456 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.576462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-oidc-ca\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.576653 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.576491 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mr5v\" (UniqueName: \"kubernetes.io/projected/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-kube-api-access-9mr5v\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.577092 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.577073 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-oidc-ca\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.578911 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.578893 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-tls-cert\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.584562 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.584538 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mr5v\" (UniqueName: \"kubernetes.io/projected/d0fceffb-f0ce-4fc5-aa28-020b4d2a461a-kube-api-access-9mr5v\") pod \"authorino-6f68bd4764-mtjzz\" (UID: \"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a\") " pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.655566 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.655526 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f68bd4764-mtjzz" Jun 10 13:04:30.789527 ip-10-0-140-124 kubenswrapper[2577]: W0610 13:04:30.789496 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0fceffb_f0ce_4fc5_aa28_020b4d2a461a.slice/crio-49a3666b99161fee34fac3538f90727ee1926dc58ad8a9e5f90cd14127c6c162 WatchSource:0}: Error finding container 49a3666b99161fee34fac3538f90727ee1926dc58ad8a9e5f90cd14127c6c162: Status 404 returned error can't find the container with id 49a3666b99161fee34fac3538f90727ee1926dc58ad8a9e5f90cd14127c6c162 Jun 10 13:04:30.790762 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.790744 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 13:04:30.792535 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.792510 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f68bd4764-mtjzz"] Jun 10 13:04:30.955487 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:30.955402 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f68bd4764-mtjzz" event={"ID":"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a","Type":"ContainerStarted","Data":"49a3666b99161fee34fac3538f90727ee1926dc58ad8a9e5f90cd14127c6c162"} Jun 10 13:04:31.963580 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:31.963547 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f68bd4764-mtjzz" event={"ID":"d0fceffb-f0ce-4fc5-aa28-020b4d2a461a","Type":"ContainerStarted","Data":"74702b1986b72200b2ef73cb7c9217dcdb24f7d9f8b18f85bab08f6b9cfb9f87"} Jun 10 13:04:31.987998 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:31.987950 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f68bd4764-mtjzz" podStartSLOduration=1.6544361570000001 podStartE2EDuration="1.987933031s" totalImagesPullingTime="333.496874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:04:31.985083272 +0000 UTC m=+987.691387463" watchObservedRunningTime="2026-06-10 13:04:31.987933031 +0000 UTC m=+987.694237220" Jun 10 13:04:32.020742 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.020711 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7545cf9777-4pftk"] Jun 10 13:04:32.020936 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.020916 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7545cf9777-4pftk" podUID="49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" containerName="authorino" containerID="cri-o://7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd" gracePeriod=30 Jun 10 13:04:32.265430 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.265390 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:04:32.391161 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.391128 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-oidc-ca\") pod \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " Jun 10 13:04:32.391351 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.391178 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwqn5\" (UniqueName: \"kubernetes.io/projected/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-kube-api-access-mwqn5\") pod \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " Jun 10 13:04:32.391351 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.391237 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-tls-cert\") pod \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\" (UID: \"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9\") " Jun 10 13:04:32.393221 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.393188 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-kube-api-access-mwqn5" (OuterVolumeSpecName: "kube-api-access-mwqn5") pod "49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" (UID: "49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9"). InnerVolumeSpecName "kube-api-access-mwqn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:04:32.395970 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.395949 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" (UID: "49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:04:32.401318 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.401298 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" (UID: "49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 13:04:32.491862 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.491825 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-tls-cert\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:32.491862 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.491858 2577 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-oidc-ca\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:32.491862 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.491869 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwqn5\" (UniqueName: \"kubernetes.io/projected/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9-kube-api-access-mwqn5\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:04:32.969491 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.969457 2577 generic.go:356] "Generic (PLEG): container finished" podID="49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" containerID="7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd" exitCode=0 Jun 10 13:04:32.969959 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.969503 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545cf9777-4pftk" Jun 10 13:04:32.969959 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.969541 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545cf9777-4pftk" event={"ID":"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9","Type":"ContainerDied","Data":"7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd"} Jun 10 13:04:32.969959 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.969589 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545cf9777-4pftk" event={"ID":"49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9","Type":"ContainerDied","Data":"9b0a93cd0682485e0a9d1525251542022020d0e8910d8450e677494d4489bccc"} Jun 10 13:04:32.969959 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.969612 2577 scope.go:117] "RemoveContainer" containerID="7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd" Jun 10 13:04:32.978457 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.978438 2577 scope.go:117] "RemoveContainer" containerID="7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd" Jun 10 13:04:32.978728 ip-10-0-140-124 kubenswrapper[2577]: E0610 13:04:32.978709 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd\": container with ID starting with 7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd not found: ID does not exist" containerID="7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd" Jun 10 13:04:32.978774 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.978738 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd"} err="failed to get container status \"7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd\": rpc error: code = NotFound desc = could not find container \"7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd\": container with ID starting with 7a44d11e77acf8cff9b9b8ae97a7bb2815e576a8b58b468f287f906ca7e159dd not found: ID does not exist" Jun 10 13:04:32.989499 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.989473 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7545cf9777-4pftk"] Jun 10 13:04:32.991555 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:32.991537 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7545cf9777-4pftk"] Jun 10 13:04:34.792560 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:04:34.792530 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" path="/var/lib/kubelet/pods/49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9/volumes" Jun 10 13:05:01.402163 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:05:01.402122 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:05:12.279857 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:05:12.279797 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:05:20.582773 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:05:20.582734 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:05:31.580576 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:05:31.580538 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:05:39.889297 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:05:39.889262 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:05:50.288665 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:05:50.288580 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:06:21.775320 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:06:21.775284 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:06:26.296477 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:06:26.296444 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:07:05.393691 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:07:05.393653 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:07:21.887130 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:07:21.887040 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:07:59.935863 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:07:59.935826 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:08:04.835554 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:08:04.835524 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:08:04.836312 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:08:04.836297 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:08:17.390697 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:08:17.390659 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:08:31.499052 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:08:31.499016 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:08:47.687041 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:08:47.687003 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:08:50.583339 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:08:50.583301 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:09:27.389353 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:09:27.389313 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:09:30.492707 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:09:30.492673 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:09:52.689107 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:09:52.689067 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:10:02.087424 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:10:02.087385 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:10:18.383412 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:10:18.383327 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:10:26.891328 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:10:26.891287 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:10:44.176554 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:10:44.176515 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:10:52.531541 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:10:52.531501 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:11:25.096621 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:11:25.096573 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:11:33.794855 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:11:33.794817 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:11:42.884877 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:11:42.884834 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:11:50.415947 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:11:50.415864 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:11:58.890752 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:11:58.890712 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:12:07.382335 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:12:07.382299 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:12:10.489506 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:12:10.489472 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:12:16.086781 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:12:16.086743 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:12:27.086845 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:12:27.086790 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:13:04.867517 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:04.867491 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:13:04.868038 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:04.867495 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:13:05.406221 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:05.406187 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:13:07.482865 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:07.482826 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:13:15.085957 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:15.085921 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:13:18.384415 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:18.384332 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:13:26.526039 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:26.525998 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:13:34.783448 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:34.783410 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:13:44.878479 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:44.878440 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:13:52.890540 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:13:52.890502 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:14:01.676524 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:14:01.676489 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:14:10.398455 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:14:10.398417 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:14:19.084994 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:14:19.084958 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:14:27.185318 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:14:27.185280 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:14:30.285202 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:14:30.285164 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:14:36.877325 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:14:36.877288 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:14:45.683233 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:14:45.683196 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:14:54.285126 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:14:54.285032 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:15:00.153117 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.153077 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k"] Jun 10 13:15:00.153521 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.153381 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" containerName="authorino" Jun 10 13:15:00.153521 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.153392 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" containerName="authorino" Jun 10 13:15:00.153521 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.153445 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="49fa1aa7-ec20-45f5-a9ca-a3bf40f92ad9" containerName="authorino" Jun 10 13:15:00.156234 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.156218 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" Jun 10 13:15:00.180305 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.180266 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k"] Jun 10 13:15:00.224339 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.224295 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vpzf\" (UniqueName: \"kubernetes.io/projected/4ab59221-dd51-4c16-b7e8-534a221dc604-kube-api-access-6vpzf\") pod \"maas-api-key-cleanup-29684955-ptw4k\" (UID: \"4ab59221-dd51-4c16-b7e8-534a221dc604\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" Jun 10 13:15:00.325570 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.325528 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vpzf\" (UniqueName: \"kubernetes.io/projected/4ab59221-dd51-4c16-b7e8-534a221dc604-kube-api-access-6vpzf\") pod \"maas-api-key-cleanup-29684955-ptw4k\" (UID: \"4ab59221-dd51-4c16-b7e8-534a221dc604\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" Jun 10 13:15:00.334903 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.334870 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vpzf\" (UniqueName: \"kubernetes.io/projected/4ab59221-dd51-4c16-b7e8-534a221dc604-kube-api-access-6vpzf\") pod \"maas-api-key-cleanup-29684955-ptw4k\" (UID: \"4ab59221-dd51-4c16-b7e8-534a221dc604\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" Jun 10 13:15:00.465629 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.465533 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" Jun 10 13:15:00.603553 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.603529 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k"] Jun 10 13:15:00.605481 ip-10-0-140-124 kubenswrapper[2577]: W0610 13:15:00.605453 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab59221_dd51_4c16_b7e8_534a221dc604.slice/crio-fd967b96054d048c96482e73b46476439156e000e307280c53330862d953e14f WatchSource:0}: Error finding container fd967b96054d048c96482e73b46476439156e000e307280c53330862d953e14f: Status 404 returned error can't find the container with id fd967b96054d048c96482e73b46476439156e000e307280c53330862d953e14f Jun 10 13:15:00.607446 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:00.607431 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 13:15:01.092736 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:01.092701 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" event={"ID":"4ab59221-dd51-4c16-b7e8-534a221dc604","Type":"ContainerStarted","Data":"fd967b96054d048c96482e73b46476439156e000e307280c53330862d953e14f"} Jun 10 13:15:02.890370 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:02.890330 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:15:04.110677 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:04.110637 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" event={"ID":"4ab59221-dd51-4c16-b7e8-534a221dc604","Type":"ContainerStarted","Data":"1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34"} Jun 10 13:15:04.127140 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:04.127088 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" podStartSLOduration=1.920090111 podStartE2EDuration="4.127071927s" totalImagesPullingTime="2.206981816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:15:04.125546113 +0000 UTC m=+1619.831850303" watchObservedRunningTime="2026-06-10 13:15:04.127071927 +0000 UTC m=+1619.833376115" Jun 10 13:15:11.489315 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:11.489273 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:15:19.609249 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:19.609211 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:15:29.812142 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:29.812109 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:15:37.882104 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:15:37.882066 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:16:00.013039 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:00.013002 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k"] Jun 10 13:16:00.013435 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:00.013208 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" podUID="4ab59221-dd51-4c16-b7e8-534a221dc604" containerName="cleanup" containerID="cri-o://1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34" gracePeriod=30 Jun 10 13:16:30.183613 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.183582 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" Jun 10 13:16:30.291586 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.291502 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vpzf\" (UniqueName: \"kubernetes.io/projected/4ab59221-dd51-4c16-b7e8-534a221dc604-kube-api-access-6vpzf\") pod \"4ab59221-dd51-4c16-b7e8-534a221dc604\" (UID: \"4ab59221-dd51-4c16-b7e8-534a221dc604\") " Jun 10 13:16:30.293640 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.293603 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab59221-dd51-4c16-b7e8-534a221dc604-kube-api-access-6vpzf" (OuterVolumeSpecName: "kube-api-access-6vpzf") pod "4ab59221-dd51-4c16-b7e8-534a221dc604" (UID: "4ab59221-dd51-4c16-b7e8-534a221dc604"). InnerVolumeSpecName "kube-api-access-6vpzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:16:30.392061 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.392024 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vpzf\" (UniqueName: \"kubernetes.io/projected/4ab59221-dd51-4c16-b7e8-534a221dc604-kube-api-access-6vpzf\") on node \"ip-10-0-140-124.ec2.internal\" DevicePath \"\"" Jun 10 13:16:30.543579 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.543488 2577 generic.go:356] "Generic (PLEG): container finished" podID="4ab59221-dd51-4c16-b7e8-534a221dc604" containerID="1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34" exitCode=137 Jun 10 13:16:30.543579 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.543538 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" Jun 10 13:16:30.543962 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.543575 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" event={"ID":"4ab59221-dd51-4c16-b7e8-534a221dc604","Type":"ContainerDied","Data":"1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34"} Jun 10 13:16:30.543962 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.543631 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k" event={"ID":"4ab59221-dd51-4c16-b7e8-534a221dc604","Type":"ContainerDied","Data":"fd967b96054d048c96482e73b46476439156e000e307280c53330862d953e14f"} Jun 10 13:16:30.543962 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.543658 2577 scope.go:117] "RemoveContainer" containerID="1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34" Jun 10 13:16:30.552641 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.552622 2577 scope.go:117] "RemoveContainer" containerID="1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34" Jun 10 13:16:30.552973 ip-10-0-140-124 kubenswrapper[2577]: E0610 13:16:30.552952 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34\": container with ID starting with 1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34 not found: ID does not exist" containerID="1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34" Jun 10 13:16:30.553039 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.552985 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34"} err="failed to get container status \"1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34\": rpc error: code = NotFound desc = could not find container \"1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34\": container with ID starting with 1a27594879a11144a1b957247c035a015de1f71c3c51b67e6fa82be913a06a34 not found: ID does not exist" Jun 10 13:16:30.569361 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.569329 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k"] Jun 10 13:16:30.572550 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.572528 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684955-ptw4k"] Jun 10 13:16:30.791152 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:16:30.791122 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab59221-dd51-4c16-b7e8-534a221dc604" path="/var/lib/kubelet/pods/4ab59221-dd51-4c16-b7e8-534a221dc604/volumes" Jun 10 13:17:55.783507 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:17:55.783473 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:18:00.780961 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:18:00.780921 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:18:04.899846 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:18:04.899798 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:18:04.900282 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:18:04.900249 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:18:26.681820 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:18:26.681765 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:18:31.381976 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:18:31.381935 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:18:40.897987 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:18:40.897952 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:18:51.884037 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:18:51.884000 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:19:00.192725 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:19:00.192683 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:19:11.191302 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:19:11.191256 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:19:19.683260 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:19:19.683177 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:19:30.488168 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:19:30.488127 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:19:39.197748 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:19:39.197701 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:19:49.785708 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:19:49.785666 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:19:58.889936 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:19:58.889903 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:20:03.199868 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:20:03.199831 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:21:32.379537 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:21:32.379493 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:22:15.385392 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:22:15.385359 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:22:23.786271 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:22:23.786190 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:22:32.483026 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:22:32.482980 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:22:41.379343 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:22:41.379304 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:22:49.886276 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:22:49.886242 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:23:00.804482 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:00.804442 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:23:04.930940 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:04.930916 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:23:04.932590 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:04.932571 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqwjk_18ff83ff-de0e-418c-bab6-cdbf795df6ce/ovn-acl-logging/0.log" Jun 10 13:23:09.595818 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:09.595770 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:23:17.094689 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:17.094650 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:23:25.835225 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:25.835186 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:23:34.207616 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:34.207575 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:23:42.634831 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:42.634784 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:23:52.918644 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:23:52.918561 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:24:11.035315 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:24:11.035276 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:24:19.506892 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:24:19.506857 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:24:28.308765 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:24:28.308729 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:24:36.846978 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:24:36.846929 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:24:53.830554 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:24:53.830520 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:25:02.210402 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:25:02.210364 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:25:11.088220 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:25:11.088180 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:25:19.688918 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:25:19.688825 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:25:28.489752 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:25:28.489714 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:25:36.992496 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:25:36.992460 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:25:46.096923 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:25:46.096889 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:25:57.405253 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:25:57.405217 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:26:06.614166 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:06.614122 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:26:17.695109 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:17.695072 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:26:26.895397 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:26.895359 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:26:34.287952 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:34.287909 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-spdfr"] Jun 10 13:26:42.174872 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:42.174834 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f68bd4764-mtjzz_d0fceffb-f0ce-4fc5-aa28-020b4d2a461a/authorino/0.log" Jun 10 13:26:47.289941 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:47.289907 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-skxnl_bbf88196-c7ea-4013-ba18-c17da3bce1ae/server/0.log" Jun 10 13:26:47.411026 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:47.410992 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-fz8sb_d1e0b919-f45a-444a-8631-703036d2bfce/manager/0.log" Jun 10 13:26:47.524622 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:47.524592 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f4c98d49d-7vk7q_efcab33d-1f6b-4a52-8141-f9228f2b83bc/manager/0.log" Jun 10 13:26:47.533913 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:47.533891 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f4c98d49d-7vk7q_efcab33d-1f6b-4a52-8141-f9228f2b83bc/copy-manifests/0.log" Jun 10 13:26:48.581099 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.581018 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw_b613471e-a5ab-4552-9d9d-387564bd9280/util/0.log" Jun 10 13:26:48.588023 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.587997 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw_b613471e-a5ab-4552-9d9d-387564bd9280/pull/0.log" Jun 10 13:26:48.594433 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.594411 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw_b613471e-a5ab-4552-9d9d-387564bd9280/extract/0.log" Jun 10 13:26:48.709076 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.709042 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj_2ac72622-74c6-43f0-8796-700350e39b7a/pull/0.log" Jun 10 13:26:48.716158 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.716121 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj_2ac72622-74c6-43f0-8796-700350e39b7a/extract/0.log" Jun 10 13:26:48.722689 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.722670 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj_2ac72622-74c6-43f0-8796-700350e39b7a/util/0.log" Jun 10 13:26:48.832119 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.832040 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d_f07091df-a21f-4e5f-bb49-6e9ed46c25c1/util/0.log" Jun 10 13:26:48.838107 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.838082 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d_f07091df-a21f-4e5f-bb49-6e9ed46c25c1/pull/0.log" Jun 10 13:26:48.847021 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.846996 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d_f07091df-a21f-4e5f-bb49-6e9ed46c25c1/extract/0.log" Jun 10 13:26:48.965067 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.965034 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb_f7c0372d-987d-4711-bbbe-bb126c2d90dd/util/0.log" Jun 10 13:26:48.974555 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.974532 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb_f7c0372d-987d-4711-bbbe-bb126c2d90dd/pull/0.log" Jun 10 13:26:48.981379 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:48.981356 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb_f7c0372d-987d-4711-bbbe-bb126c2d90dd/extract/0.log" Jun 10 13:26:49.097869 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:49.097788 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f68bd4764-mtjzz_d0fceffb-f0ce-4fc5-aa28-020b4d2a461a/authorino/0.log" Jun 10 13:26:49.430433 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:49.430407 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6hppm_51b83f0f-073b-4157-a5b2-0988a2e0213e/kuadrant-console-plugin/0.log" Jun 10 13:26:49.798765 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:49.798679 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-spdfr_46022d38-d4a0-471f-9630-7ad1096ef21f/limitador/0.log" Jun 10 13:26:50.639423 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:50.639392 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796c65c65-jxm89_8500ab8f-1dd1-4216-a61b-e3a28b91655b/kube-auth-proxy/0.log" Jun 10 13:26:50.878633 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:50.878599 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-dgmbc_b724625f-9d67-4649-bc9e-73da596fdde1/payload-pre-processing/0.log" Jun 10 13:26:56.524200 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.524168 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jls6j/must-gather-z2t6d"] Jun 10 13:26:56.524565 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.524467 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ab59221-dd51-4c16-b7e8-534a221dc604" containerName="cleanup" Jun 10 13:26:56.524565 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.524478 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ab59221-dd51-4c16-b7e8-534a221dc604" containerName="cleanup" Jun 10 13:26:56.524565 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.524528 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ab59221-dd51-4c16-b7e8-534a221dc604" containerName="cleanup" Jun 10 13:26:56.527663 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.527646 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jls6j/must-gather-z2t6d" Jun 10 13:26:56.531552 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.531526 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jls6j\"/\"openshift-service-ca.crt\"" Jun 10 13:26:56.531552 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.531543 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jls6j\"/\"kube-root-ca.crt\"" Jun 10 13:26:56.531740 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.531526 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jls6j\"/\"default-dockercfg-8zlpx\"" Jun 10 13:26:56.548244 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.548208 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jls6j/must-gather-z2t6d"] Jun 10 13:26:56.570528 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.570497 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngshl\" (UniqueName: \"kubernetes.io/projected/c803664d-7173-4b84-80db-1e1a342566b5-kube-api-access-ngshl\") pod \"must-gather-z2t6d\" (UID: \"c803664d-7173-4b84-80db-1e1a342566b5\") " pod="openshift-must-gather-jls6j/must-gather-z2t6d" Jun 10 13:26:56.570690 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.570535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c803664d-7173-4b84-80db-1e1a342566b5-must-gather-output\") pod \"must-gather-z2t6d\" (UID: \"c803664d-7173-4b84-80db-1e1a342566b5\") " pod="openshift-must-gather-jls6j/must-gather-z2t6d" Jun 10 13:26:56.671166 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.671123 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngshl\" (UniqueName: \"kubernetes.io/projected/c803664d-7173-4b84-80db-1e1a342566b5-kube-api-access-ngshl\") pod \"must-gather-z2t6d\" (UID: \"c803664d-7173-4b84-80db-1e1a342566b5\") " pod="openshift-must-gather-jls6j/must-gather-z2t6d" Jun 10 13:26:56.671166 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.671167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c803664d-7173-4b84-80db-1e1a342566b5-must-gather-output\") pod \"must-gather-z2t6d\" (UID: \"c803664d-7173-4b84-80db-1e1a342566b5\") " pod="openshift-must-gather-jls6j/must-gather-z2t6d" Jun 10 13:26:56.671471 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.671457 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c803664d-7173-4b84-80db-1e1a342566b5-must-gather-output\") pod \"must-gather-z2t6d\" (UID: \"c803664d-7173-4b84-80db-1e1a342566b5\") " pod="openshift-must-gather-jls6j/must-gather-z2t6d" Jun 10 13:26:56.682158 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.682132 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngshl\" (UniqueName: \"kubernetes.io/projected/c803664d-7173-4b84-80db-1e1a342566b5-kube-api-access-ngshl\") pod \"must-gather-z2t6d\" (UID: \"c803664d-7173-4b84-80db-1e1a342566b5\") " pod="openshift-must-gather-jls6j/must-gather-z2t6d" Jun 10 13:26:56.836658 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.836560 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jls6j/must-gather-z2t6d" Jun 10 13:26:56.968400 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.968371 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jls6j/must-gather-z2t6d"] Jun 10 13:26:56.969584 ip-10-0-140-124 kubenswrapper[2577]: W0610 13:26:56.969558 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc803664d_7173_4b84_80db_1e1a342566b5.slice/crio-4b0356afa5637609495e6fed9bac10a99ebe66e184cc645495c762211c0818ed WatchSource:0}: Error finding container 4b0356afa5637609495e6fed9bac10a99ebe66e184cc645495c762211c0818ed: Status 404 returned error can't find the container with id 4b0356afa5637609495e6fed9bac10a99ebe66e184cc645495c762211c0818ed Jun 10 13:26:56.971326 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:56.971310 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 13:26:57.652044 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:57.651984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jls6j/must-gather-z2t6d" event={"ID":"c803664d-7173-4b84-80db-1e1a342566b5","Type":"ContainerStarted","Data":"4b0356afa5637609495e6fed9bac10a99ebe66e184cc645495c762211c0818ed"} Jun 10 13:26:58.661401 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:58.661364 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jls6j/must-gather-z2t6d" event={"ID":"c803664d-7173-4b84-80db-1e1a342566b5","Type":"ContainerStarted","Data":"997268eff533fab1aa0155c0a2641cc585f9719495a4d61d83fef6c99bef7eed"} Jun 10 13:26:58.661796 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:58.661414 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jls6j/must-gather-z2t6d" event={"ID":"c803664d-7173-4b84-80db-1e1a342566b5","Type":"ContainerStarted","Data":"7ebd0c16ce1590edede8dc36d059a320b6815f99da829f246a5925bd820a6d3b"} Jun 10 13:26:58.680232 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:58.680172 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jls6j/must-gather-z2t6d" podStartSLOduration=2.037162996 podStartE2EDuration="2.68015472s" totalImagesPullingTime="642.991724ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:26:58.67894329 +0000 UTC m=+2334.385247478" watchObservedRunningTime="2026-06-10 13:26:58.68015472 +0000 UTC m=+2334.386458942" Jun 10 13:26:59.463233 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:59.463198 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k5ktc_cbeeb78c-7022-4a4e-8834-767fef6bb920/global-pull-secret-syncer/0.log" Jun 10 13:26:59.584837 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:59.584788 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j8bc8_6abcc7d2-5450-49f2-9ffb-e7e297e3e9ab/konnectivity-agent/0.log" Jun 10 13:26:59.655648 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:26:59.655620 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-124.ec2.internal_465107ac2ceb7852c065a227546cfcdf/haproxy/0.log" Jun 10 13:27:03.648627 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.648564 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw_b613471e-a5ab-4552-9d9d-387564bd9280/extract/0.log" Jun 10 13:27:03.681171 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.681109 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw_b613471e-a5ab-4552-9d9d-387564bd9280/util/0.log" Jun 10 13:27:03.707237 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.707206 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759skkrw_b613471e-a5ab-4552-9d9d-387564bd9280/pull/0.log" Jun 10 13:27:03.760167 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.760141 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj_2ac72622-74c6-43f0-8796-700350e39b7a/extract/0.log" Jun 10 13:27:03.782182 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.782156 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj_2ac72622-74c6-43f0-8796-700350e39b7a/util/0.log" Jun 10 13:27:03.805278 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.805244 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrmqj_2ac72622-74c6-43f0-8796-700350e39b7a/pull/0.log" Jun 10 13:27:03.834224 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.834194 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d_f07091df-a21f-4e5f-bb49-6e9ed46c25c1/extract/0.log" Jun 10 13:27:03.872928 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.872901 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d_f07091df-a21f-4e5f-bb49-6e9ed46c25c1/util/0.log" Jun 10 13:27:03.898156 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.898122 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732vq6d_f07091df-a21f-4e5f-bb49-6e9ed46c25c1/pull/0.log" Jun 10 13:27:03.936789 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.936704 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb_f7c0372d-987d-4711-bbbe-bb126c2d90dd/extract/0.log" Jun 10 13:27:03.956557 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.956523 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb_f7c0372d-987d-4711-bbbe-bb126c2d90dd/util/0.log" Jun 10 13:27:03.982997 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:03.982957 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j4hsb_f7c0372d-987d-4711-bbbe-bb126c2d90dd/pull/0.log" Jun 10 13:27:04.230928 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:04.230771 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f68bd4764-mtjzz_d0fceffb-f0ce-4fc5-aa28-020b4d2a461a/authorino/0.log" Jun 10 13:27:04.321775 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:04.321737 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6hppm_51b83f0f-073b-4157-a5b2-0988a2e0213e/kuadrant-console-plugin/0.log" Jun 10 13:27:04.475420 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:04.475391 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-spdfr_46022d38-d4a0-471f-9630-7ad1096ef21f/limitador/0.log" Jun 10 13:27:06.688025 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:06.687987 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d98576f89-87kbh_c115d7e5-d1fd-4255-a140-35f67dc5f3d0/metrics-server/0.log" Jun 10 13:27:06.919448 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:06.919409 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xn9xr_d116ca5f-e850-497f-a8ac-48f845f4cc1f/node-exporter/0.log" Jun 10 13:27:06.945886 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:06.945853 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xn9xr_d116ca5f-e850-497f-a8ac-48f845f4cc1f/kube-rbac-proxy/0.log" Jun 10 13:27:06.968741 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:06.968709 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xn9xr_d116ca5f-e850-497f-a8ac-48f845f4cc1f/init-textfile/0.log" Jun 10 13:27:10.515483 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:10.515411 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c4786f77f-r9pdp_98c990c9-0e68-41da-8c00-c950429a31c5/console/0.log" Jun 10 13:27:11.772396 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.772359 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r"] Jun 10 13:27:11.778578 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.778545 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.785733 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.785700 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r"] Jun 10 13:27:11.814059 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.814001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-proc\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.814244 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.814085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-lib-modules\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.814323 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.814255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-podres\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.814435 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.814405 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvjcp\" (UniqueName: \"kubernetes.io/projected/bb9803b6-198f-4ffa-8240-d90197dccca3-kube-api-access-xvjcp\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.815487 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.815466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-sys\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.916998 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.916962 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvjcp\" (UniqueName: \"kubernetes.io/projected/bb9803b6-198f-4ffa-8240-d90197dccca3-kube-api-access-xvjcp\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.917270 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.917253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-sys\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.917396 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.917382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-proc\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.917511 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.917484 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-sys\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.917678 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.917657 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-proc\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.917755 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.917615 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-lib-modules\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.917831 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.917488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-lib-modules\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.917896 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.917838 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-podres\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.917977 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.917958 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb9803b6-198f-4ffa-8240-d90197dccca3-podres\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:11.925555 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:11.925534 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvjcp\" (UniqueName: \"kubernetes.io/projected/bb9803b6-198f-4ffa-8240-d90197dccca3-kube-api-access-xvjcp\") pod \"perf-node-gather-daemonset-76f6r\" (UID: \"bb9803b6-198f-4ffa-8240-d90197dccca3\") " pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:12.099617 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.099520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:12.270102 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.268019 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r"] Jun 10 13:27:12.568718 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.568659 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v78tb_55dd8921-54a8-4d4f-beb9-ffd85f02ee5c/dns/0.log" Jun 10 13:27:12.591382 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.591355 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v78tb_55dd8921-54a8-4d4f-beb9-ffd85f02ee5c/kube-rbac-proxy/0.log" Jun 10 13:27:12.664397 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.664368 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wqrsz_1cd4f44a-0a1e-4710-a68c-be170c527c22/dns-node-resolver/0.log" Jun 10 13:27:12.796177 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.796143 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:12.796548 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.796225 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" event={"ID":"bb9803b6-198f-4ffa-8240-d90197dccca3","Type":"ContainerStarted","Data":"7ce94e4cbc6425aec1ca1f4505f6f2b00a6700217f2ea4aebcab296504142aef"} Jun 10 13:27:12.796548 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.796258 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" event={"ID":"bb9803b6-198f-4ffa-8240-d90197dccca3","Type":"ContainerStarted","Data":"944e8ad4f751f635c6ffa3e0f6434eb4cccc048d53f50749b0f1c86031addd79"} Jun 10 13:27:12.811541 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:12.811485 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" podStartSLOduration=1.811467325 podStartE2EDuration="1.811467325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 13:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:27:12.810321972 +0000 UTC m=+2348.516626162" watchObservedRunningTime="2026-06-10 13:27:12.811467325 +0000 UTC m=+2348.517771515" Jun 10 13:27:13.267959 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:13.267928 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k7tz2_8136e2a8-4ddc-46fa-b368-c6803750c135/node-ca/0.log" Jun 10 13:27:14.355783 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:14.355749 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796c65c65-jxm89_8500ab8f-1dd1-4216-a61b-e3a28b91655b/kube-auth-proxy/0.log" Jun 10 13:27:14.444435 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:14.444403 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-dgmbc_b724625f-9d67-4649-bc9e-73da596fdde1/payload-pre-processing/0.log" Jun 10 13:27:15.209200 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:15.209171 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sbvn9_513a89d3-0aa0-4f3a-8acb-ce4f074ac9f1/serve-healthcheck-canary/0.log" Jun 10 13:27:15.845775 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:15.845748 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w2d49_503b2798-f51f-4e92-9b18-eb2187fc6d7c/kube-rbac-proxy/0.log" Jun 10 13:27:15.866894 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:15.866865 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w2d49_503b2798-f51f-4e92-9b18-eb2187fc6d7c/exporter/0.log" Jun 10 13:27:15.890386 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:15.890339 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w2d49_503b2798-f51f-4e92-9b18-eb2187fc6d7c/extractor/0.log" Jun 10 13:27:18.531897 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:18.531870 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-skxnl_bbf88196-c7ea-4013-ba18-c17da3bce1ae/server/0.log" Jun 10 13:27:18.592755 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:18.592704 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-fz8sb_d1e0b919-f45a-444a-8631-703036d2bfce/manager/0.log" Jun 10 13:27:18.619943 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:18.619907 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f4c98d49d-7vk7q_efcab33d-1f6b-4a52-8141-f9228f2b83bc/manager/0.log" Jun 10 13:27:18.642428 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:18.642399 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f4c98d49d-7vk7q_efcab33d-1f6b-4a52-8141-f9228f2b83bc/copy-manifests/0.log" Jun 10 13:27:18.811064 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:18.810985 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jls6j/perf-node-gather-daemonset-76f6r" Jun 10 13:27:20.346617 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:20.346571 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-546fcb9487-7rdgq_07f5937b-fb14-4a7a-a4d4-c256390ea742/manager/0.log" Jun 10 13:27:24.855511 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:24.855482 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796c65c65-jxm89_8500ab8f-1dd1-4216-a61b-e3a28b91655b/kube-auth-proxy/0.log" Jun 10 13:27:25.004521 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:25.004489 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-dgmbc_b724625f-9d67-4649-bc9e-73da596fdde1/payload-pre-processing/0.log" Jun 10 13:27:26.307293 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:26.307267 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796c65c65-jxm89_8500ab8f-1dd1-4216-a61b-e3a28b91655b/kube-auth-proxy/0.log" Jun 10 13:27:26.382656 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:26.382626 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-dgmbc_b724625f-9d67-4649-bc9e-73da596fdde1/payload-pre-processing/0.log" Jun 10 13:27:28.205587 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.205560 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796c65c65-jxm89_8500ab8f-1dd1-4216-a61b-e3a28b91655b/kube-auth-proxy/0.log" Jun 10 13:27:28.272215 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.272186 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-dgmbc_b724625f-9d67-4649-bc9e-73da596fdde1/payload-pre-processing/0.log" Jun 10 13:27:28.296369 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.296340 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bz7mt_213701d6-76f7-4d6e-8012-6918d482c2c9/kube-multus-additional-cni-plugins/0.log" Jun 10 13:27:28.319896 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.319853 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bz7mt_213701d6-76f7-4d6e-8012-6918d482c2c9/egress-router-binary-copy/0.log" Jun 10 13:27:28.339692 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.339664 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bz7mt_213701d6-76f7-4d6e-8012-6918d482c2c9/cni-plugins/0.log" Jun 10 13:27:28.363736 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.363712 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bz7mt_213701d6-76f7-4d6e-8012-6918d482c2c9/bond-cni-plugin/0.log" Jun 10 13:27:28.389419 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.389385 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bz7mt_213701d6-76f7-4d6e-8012-6918d482c2c9/routeoverride-cni/0.log" Jun 10 13:27:28.417009 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.416981 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bz7mt_213701d6-76f7-4d6e-8012-6918d482c2c9/whereabouts-cni-bincopy/0.log" Jun 10 13:27:28.445157 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.445129 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bz7mt_213701d6-76f7-4d6e-8012-6918d482c2c9/whereabouts-cni/0.log" Jun 10 13:27:28.701825 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.701782 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z22dk_16a5c16f-7a51-4717-b623-fb1f604add4c/kube-multus/0.log" Jun 10 13:27:28.783780 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.783749 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gcbmf_1d88de4b-5158-4a97-b796-74a12f088e70/network-metrics-daemon/0.log" Jun 10 13:27:28.807584 ip-10-0-140-124 kubenswrapper[2577]: I0610 13:27:28.806797 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gcbmf_1d88de4b-5158-4a97-b796-74a12f088e70/kube-rbac-proxy/0.log"