Jun 03 13:44:50.563572 ip-10-0-143-102 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 03 13:44:50.563584 ip-10-0-143-102 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 03 13:44:50.563590 ip-10-0-143-102 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 03 13:44:50.563833 ip-10-0-143-102 systemd[1]: Failed to start Kubernetes Kubelet. Jun 03 13:45:00.615832 ip-10-0-143-102 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 03 13:45:00.615856 ip-10-0-143-102 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 76c17945e6434726a42ccf1dd8890a5a -- Jun 03 13:47:22.119169 ip-10-0-143-102 systemd[1]: Starting Kubernetes Kubelet... Jun 03 13:47:22.522181 ip-10-0-143-102 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 13:47:22.522181 ip-10-0-143-102 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 13:47:22.522181 ip-10-0-143-102 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 13:47:22.522181 ip-10-0-143-102 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 13:47:22.522181 ip-10-0-143-102 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 13:47:22.523760 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.523672 2573 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530713 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530733 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530737 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530740 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530743 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530746 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530750 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530753 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:47:22.530742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530755 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530758 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530761 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530764 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530766 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530769 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530771 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530774 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530777 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530780 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530782 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530785 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530788 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530791 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530794 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530796 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530799 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530802 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530804 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530807 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:47:22.531073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530809 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530812 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530814 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530817 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530820 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530822 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530825 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530827 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530830 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530832 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530834 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530837 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530839 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530842 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530845 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530849 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530851 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530855 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530857 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530860 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:47:22.531555 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530862 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530865 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530868 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530870 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530873 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530876 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530879 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530901 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530908 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530913 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530917 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530921 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530924 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530927 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530930 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530933 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530935 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530938 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530941 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:47:22.532123 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530943 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530946 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530948 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530951 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530954 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530957 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530961 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530964 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530967 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530970 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530973 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530976 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530979 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530982 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530985 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530987 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530990 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530992 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.530995 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531377 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:47:22.532665 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531382 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531385 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531388 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531391 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531393 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531396 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531398 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531401 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531404 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531406 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531410 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531412 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531415 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531418 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531420 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531423 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531425 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531428 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531431 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:47:22.533173 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531435 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531438 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531440 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531443 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531446 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531449 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531451 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531454 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531456 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531459 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531461 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531464 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531466 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531469 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531471 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531476 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531479 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531482 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531485 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531488 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:47:22.533632 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531490 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531493 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531496 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531498 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531500 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531503 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531505 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531508 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531510 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531512 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531515 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531519 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531523 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531527 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531529 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531532 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531534 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531538 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531540 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531543 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:47:22.534150 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531545 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531548 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531550 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531552 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531555 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531557 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531560 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531563 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531566 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531569 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531572 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531574 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531577 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531579 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531582 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531585 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531588 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531590 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531592 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:47:22.534670 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531595 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531597 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531600 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531602 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531605 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531608 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.531610 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531688 2573 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531696 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531704 2573 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531708 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531714 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531717 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531722 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531726 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531729 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531733 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531736 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531739 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531743 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531745 2573 flags.go:64] FLAG: --cgroup-root="" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531749 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531752 2573 flags.go:64] FLAG: --client-ca-file="" Jun 03 13:47:22.535148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531755 2573 flags.go:64] FLAG: --cloud-config="" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531758 2573 flags.go:64] FLAG: --cloud-provider="external" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531761 2573 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531765 2573 flags.go:64] FLAG: --cluster-domain="" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531768 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531771 2573 flags.go:64] FLAG: --config-dir="" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531774 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531777 2573 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531781 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531784 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531787 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531790 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531793 2573 flags.go:64] FLAG: --contention-profiling="false" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531799 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531802 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531805 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531809 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531813 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531816 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531819 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531822 2573 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531825 2573 flags.go:64] FLAG: --enable-server="true" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531828 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531840 2573 flags.go:64] FLAG: --event-burst="100" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531844 2573 flags.go:64] FLAG: --event-qps="50" Jun 03 13:47:22.535723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531847 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531850 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531853 2573 flags.go:64] FLAG: --eviction-hard="" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531860 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531863 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531867 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531870 2573 flags.go:64] FLAG: --eviction-soft="" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531873 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531876 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531879 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531896 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531907 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531912 2573 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531917 2573 flags.go:64] FLAG: --feature-gates="" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531922 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531925 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531928 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531931 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531934 2573 flags.go:64] FLAG: --healthz-port="10248" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531937 2573 flags.go:64] FLAG: --help="false" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531942 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-143-102.ec2.internal" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531945 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531947 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 13:47:22.536364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531951 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531954 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531957 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531960 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531963 2573 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531966 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531969 2573 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531972 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531975 2573 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531978 2573 flags.go:64] FLAG: --kube-reserved="" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531981 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531984 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531987 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531990 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531993 2573 flags.go:64] FLAG: --lock-file="" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531996 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.531999 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532002 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532007 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532009 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532012 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532015 2573 flags.go:64] FLAG: --logging-format="text" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532018 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532021 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 13:47:22.536966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532024 2573 flags.go:64] FLAG: --manifest-url="" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532027 2573 flags.go:64] FLAG: --manifest-url-header="" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532031 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532034 2573 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532038 2573 flags.go:64] FLAG: --max-pods="110" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532042 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532045 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532048 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532051 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532054 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532057 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532060 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532068 2573 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532071 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532074 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532078 2573 flags.go:64] FLAG: --pod-cidr="" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532081 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532086 2573 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532089 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532092 2573 flags.go:64] FLAG: --pods-per-core="0" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532095 2573 flags.go:64] FLAG: --port="10250" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532098 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532101 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-090b13b35afcb0d8d" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532105 2573 flags.go:64] FLAG: --qos-reserved="" Jun 03 13:47:22.537555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532107 2573 flags.go:64] FLAG: --read-only-port="10255" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532110 2573 flags.go:64] FLAG: --register-node="true" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532114 2573 flags.go:64] FLAG: --register-schedulable="true" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532117 2573 flags.go:64] FLAG: --register-with-taints="" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532120 2573 flags.go:64] FLAG: --registry-burst="10" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532123 2573 flags.go:64] FLAG: --registry-qps="5" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532126 2573 flags.go:64] FLAG: --reserved-cpus="" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532129 2573 flags.go:64] FLAG: --reserved-memory="" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532133 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532136 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532139 2573 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532142 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532145 2573 flags.go:64] FLAG: --runonce="false" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532147 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532152 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532154 2573 flags.go:64] FLAG: --seccomp-default="false" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532157 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532160 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532163 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532166 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532169 2573 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532172 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532175 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532178 2573 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532181 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532184 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 13:47:22.538197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532187 2573 flags.go:64] FLAG: --system-cgroups="" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532190 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532195 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532199 2573 flags.go:64] FLAG: --tls-cert-file="" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532201 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532205 2573 flags.go:64] FLAG: --tls-min-version="" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532208 2573 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532211 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532214 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532217 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532220 2573 flags.go:64] FLAG: --v="2" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532224 2573 flags.go:64] FLAG: --version="false" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532227 2573 flags.go:64] FLAG: --vmodule="" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532231 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532235 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532327 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532331 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532334 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532336 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532339 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532343 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532348 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532351 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:47:22.538820 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532354 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532357 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532360 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532363 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532367 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532370 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532373 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532375 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532379 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532382 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532384 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532387 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532391 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532393 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532396 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532399 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532401 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532403 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532406 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532408 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:47:22.539383 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532411 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532413 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532416 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532418 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532421 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532423 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532426 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532429 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532431 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532435 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532438 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532440 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532444 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532448 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532452 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532454 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532457 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532459 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532462 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:47:22.539953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532464 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532467 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532470 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532473 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532475 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532478 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532480 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532483 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532485 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532488 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532490 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532493 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532495 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532498 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532500 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532503 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532505 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532508 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532510 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532513 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:47:22.540437 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532515 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532518 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532522 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532525 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532528 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532530 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532533 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532537 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532539 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532542 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532544 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532547 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532549 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532552 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532554 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532557 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532560 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532563 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:47:22.540994 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.532565 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.532570 2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.539845 2573 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.539865 2573 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539932 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539938 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539941 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539944 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539946 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539950 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539953 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539956 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539958 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539962 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539964 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539967 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:47:22.541457 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539970 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539973 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539975 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539978 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539981 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539983 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539986 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539989 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539991 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539994 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539997 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.539999 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540002 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540005 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540008 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540011 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540013 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540016 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540018 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540021 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:47:22.541865 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540024 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540027 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540029 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540032 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540035 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540037 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540039 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540042 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540045 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540047 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540050 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540053 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540055 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540058 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540062 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540065 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540068 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540071 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540074 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:47:22.542362 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540077 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540080 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540083 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540086 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540088 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540091 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540094 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540098 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540101 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540104 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540107 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540110 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540113 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540116 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540119 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540122 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540124 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540127 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540130 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540132 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:47:22.542831 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540135 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540137 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540141 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540145 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540148 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540151 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540153 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540156 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540159 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540161 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540164 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540166 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540169 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540172 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540174 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:47:22.543354 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.540180 2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540276 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540281 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540284 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540287 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540290 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540293 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540296 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540299 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540302 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540305 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540308 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540311 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540313 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540316 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540319 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540323 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540326 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540329 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540332 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:47:22.543768 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540334 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540337 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540340 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540343 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540345 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540348 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540350 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540353 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540356 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540358 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540361 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540363 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540366 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540369 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540371 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540374 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540376 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540379 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540382 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540385 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:47:22.544286 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540387 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540390 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540392 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540395 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540399 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540402 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540405 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540408 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540411 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540414 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540416 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540419 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540422 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540424 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540427 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540429 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540432 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540434 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540437 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:47:22.544801 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540440 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540443 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540446 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540448 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540451 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540454 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540456 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540459 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540461 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540464 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540467 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540469 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540472 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540475 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540477 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540480 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540482 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540485 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540487 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540490 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:47:22.545355 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540492 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540495 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540498 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540500 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540503 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540505 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540508 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:22.540511 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.540516 2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.541176 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.543332 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.544314 2573 server.go:1019] "Starting client certificate rotation" Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.544406 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 13:47:22.545854 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.544438 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 13:47:22.568990 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.568970 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 13:47:22.570691 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.570677 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 13:47:22.582779 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.582757 2573 log.go:25] "Validated CRI v1 runtime API" Jun 03 13:47:22.588811 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.588794 2573 log.go:25] "Validated CRI v1 image API" Jun 03 13:47:22.590655 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.590632 2573 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 13:47:22.596621 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.596602 2573 fs.go:135] Filesystem UUIDs: map[72c8838b-35d5-434d-8877-a2e550ce313e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 81232996-3871-444e-96ee-faf125927607:/dev/nvme0n1p4] Jun 03 13:47:22.596694 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.596621 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 13:47:22.601264 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.601147 2573 manager.go:217] Machine: {Timestamp:2026-06-03 13:47:22.60064306 +0000 UTC m=+0.371331776 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3067162 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2320e2edd5808880d2093e03bfbd92 SystemUUID:ec2320e2-edd5-8088-80d2-093e03bfbd92 BootID:76c17945-e643-4726-a42c-cf1dd8890a5a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7b:f2:68:9e:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7b:f2:68:9e:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:ef:83:5b:08:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 13:47:22.601552 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.601537 2573 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 13:47:22.602361 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.602349 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 13:47:22.602447 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.602435 2573 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 13:47:22.604233 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.604201 2573 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 13:47:22.604364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.604235 2573 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 13:47:22.604406 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.604374 2573 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 13:47:22.604406 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.604382 2573 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 13:47:22.604406 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.604390 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 13:47:22.605715 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.605704 2573 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 13:47:22.607257 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.607246 2573 state_mem.go:36] "Initialized new in-memory state store" Jun 03 13:47:22.607365 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.607356 2573 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 13:47:22.609476 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.609466 2573 kubelet.go:491] "Attempting to sync node with API server" Jun 03 13:47:22.609510 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.609480 2573 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 13:47:22.609510 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.609491 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 13:47:22.609510 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.609500 2573 kubelet.go:397] "Adding apiserver pod source" Jun 03 13:47:22.609510 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.609509 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 13:47:22.611124 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.611112 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 13:47:22.611169 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.611132 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 13:47:22.614068 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.614054 2573 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 13:47:22.615869 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.615854 2573 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 13:47:22.618000 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.617981 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 13:47:22.618000 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.617998 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618005 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618011 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618019 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618025 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618031 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618037 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618043 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618048 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618057 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 13:47:22.618082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618063 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 13:47:22.618810 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618802 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 13:47:22.618864 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.618810 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 13:47:22.622820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.622804 2573 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 13:47:22.622863 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.622850 2573 server.go:1295] "Started kubelet" Jun 03 13:47:22.622972 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.622929 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 13:47:22.623040 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.622990 2573 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 13:47:22.623079 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.623065 2573 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 13:47:22.623521 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.623503 2573 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 03 13:47:22.623638 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.623618 2573 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 13:47:22.623738 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.623721 2573 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 13:47:22.623787 ip-10-0-143-102 systemd[1]: Started Kubernetes Kubelet. Jun 03 13:47:22.627458 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.626843 2573 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 13:47:22.628517 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.628496 2573 server.go:317] "Adding debug handlers to kubelet server" Jun 03 13:47:22.631472 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.631452 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6hpxp" Jun 03 13:47:22.632210 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.630953 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-102.ec2.internal.18b59643191720ac default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-102.ec2.internal,UID:ip-10-0-143-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-102.ec2.internal,},FirstTimestamp:2026-06-03 13:47:22.622820524 +0000 UTC m=+0.393509271,LastTimestamp:2026-06-03 13:47:22.622820524 +0000 UTC m=+0.393509271,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-102.ec2.internal,}" Jun 03 13:47:22.632980 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.632961 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 13:47:22.633063 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.632999 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 13:47:22.633166 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.633147 2573 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 13:47:22.633166 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.633167 2573 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 13:47:22.633282 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.633265 2573 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 13:47:22.633344 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.633333 2573 reconstruct.go:97] "Volume reconstruction finished" Jun 03 13:47:22.633344 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.633341 2573 reconciler.go:26] "Reconciler: start to sync state" Jun 03 13:47:22.633763 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.633740 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:22.633936 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.633921 2573 factory.go:55] Registering systemd factory Jun 03 13:47:22.633991 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.633939 2573 factory.go:223] Registration of the systemd container factory successfully Jun 03 13:47:22.634071 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.634043 2573 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 13:47:22.634280 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.634139 2573 factory.go:153] Registering CRI-O factory Jun 03 13:47:22.634280 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.634283 2573 factory.go:223] Registration of the crio container factory successfully Jun 03 13:47:22.634403 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.634388 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 13:47:22.634461 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.634408 2573 factory.go:103] Registering Raw factory Jun 03 13:47:22.634461 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.634426 2573 manager.go:1196] Started watching for new ooms in manager Jun 03 13:47:22.634817 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.634801 2573 manager.go:319] Starting recovery of all containers Jun 03 13:47:22.640424 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.640390 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6hpxp" Jun 03 13:47:22.643655 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.643637 2573 manager.go:324] Recovery completed Jun 03 13:47:22.645731 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.645717 2573 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:47:22.648362 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.648344 2573 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-102.ec2.internal\" not found" node="ip-10-0-143-102.ec2.internal" Jun 03 13:47:22.648569 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.648559 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:47:22.650414 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.650396 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:47:22.650468 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.650427 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:47:22.650468 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.650441 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:47:22.650905 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.650878 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 13:47:22.650967 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.650905 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 13:47:22.650967 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.650925 2573 state_mem.go:36] "Initialized new in-memory state store" Jun 03 13:47:22.653180 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.653167 2573 policy_none.go:49] "None policy: Start" Jun 03 13:47:22.653226 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.653185 2573 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 13:47:22.653226 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.653196 2573 state_mem.go:35] "Initializing new in-memory state store" Jun 03 13:47:22.691499 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.691481 2573 manager.go:341] "Starting Device Plugin manager" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.691525 2573 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.691540 2573 server.go:85] "Starting device plugin registration server" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.691797 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.691810 2573 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.691906 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.691994 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.692002 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.692546 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 13:47:22.698803 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.692582 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:22.732235 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.732201 2573 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 13:47:22.733301 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.733285 2573 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 13:47:22.733391 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.733322 2573 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 13:47:22.733391 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.733342 2573 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 13:47:22.733391 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.733352 2573 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 13:47:22.733526 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.733395 2573 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 13:47:22.736092 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.736073 2573 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:47:22.792475 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.792415 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:47:22.793559 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.793541 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:47:22.793647 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.793570 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:47:22.793647 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.793580 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:47:22.793647 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.793603 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-102.ec2.internal" Jun 03 13:47:22.801927 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.801908 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-102.ec2.internal" Jun 03 13:47:22.802000 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.801934 2573 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-102.ec2.internal\": node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:22.818877 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.818853 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:22.834005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.833985 2573 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal"] Jun 03 13:47:22.834082 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.834072 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:47:22.835758 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.835742 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:47:22.835824 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.835769 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:47:22.835824 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.835779 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:47:22.836925 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.836913 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:47:22.837079 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.837065 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:22.837140 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.837096 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:47:22.837550 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.837534 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:47:22.837632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.837552 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:47:22.837632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.837563 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:47:22.837632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.837574 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:47:22.837720 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.837574 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:47:22.837720 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.837711 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:47:22.838674 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.838659 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" Jun 03 13:47:22.838807 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.838682 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:47:22.839252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.839241 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:47:22.839336 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.839264 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:47:22.839336 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:22.839276 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:47:22.863469 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.863446 2573 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-102.ec2.internal\" not found" node="ip-10-0-143-102.ec2.internal" Jun 03 13:47:22.867731 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.867718 2573 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-102.ec2.internal\" not found" node="ip-10-0-143-102.ec2.internal" Jun 03 13:47:22.919851 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:22.919834 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.020395 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.020361 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.034683 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.034655 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab071b712deb3518b2ecc312d28f3334-config\") pod \"kube-apiserver-proxy-ip-10-0-143-102.ec2.internal\" (UID: \"ab071b712deb3518b2ecc312d28f3334\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.034779 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.034691 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ed4570c8dc1824eee4a39cfb0b987e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"8ed4570c8dc1824eee4a39cfb0b987e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.034779 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.034717 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed4570c8dc1824eee4a39cfb0b987e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"8ed4570c8dc1824eee4a39cfb0b987e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.121130 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.121056 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.135340 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.135313 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ed4570c8dc1824eee4a39cfb0b987e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"8ed4570c8dc1824eee4a39cfb0b987e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.135423 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.135350 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed4570c8dc1824eee4a39cfb0b987e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"8ed4570c8dc1824eee4a39cfb0b987e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.135423 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.135377 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab071b712deb3518b2ecc312d28f3334-config\") pod \"kube-apiserver-proxy-ip-10-0-143-102.ec2.internal\" (UID: \"ab071b712deb3518b2ecc312d28f3334\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.135505 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.135417 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ed4570c8dc1824eee4a39cfb0b987e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"8ed4570c8dc1824eee4a39cfb0b987e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.135505 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.135423 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed4570c8dc1824eee4a39cfb0b987e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"8ed4570c8dc1824eee4a39cfb0b987e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.135505 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.135423 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab071b712deb3518b2ecc312d28f3334-config\") pod \"kube-apiserver-proxy-ip-10-0-143-102.ec2.internal\" (UID: \"ab071b712deb3518b2ecc312d28f3334\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.165429 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.165410 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.171026 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.171008 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" Jun 03 13:47:23.221653 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.221615 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.322128 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.322090 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.422648 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.422556 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.523085 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.523056 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.544492 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.544469 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 13:47:23.544632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.544613 2573 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 13:47:23.544682 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.544636 2573 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 13:47:23.624116 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.624081 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.633609 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.633589 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 13:47:23.642561 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.642528 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 13:42:22 +0000 UTC" deadline="2028-03-14 13:55:02.32873713 +0000 UTC" Jun 03 13:47:23.642561 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.642552 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15600h7m38.686187419s" Jun 03 13:47:23.643522 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.643509 2573 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 13:47:23.663661 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.663641 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4d2w5" Jun 03 13:47:23.671604 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.671577 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4d2w5" Jun 03 13:47:23.724951 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.724925 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.734692 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:23.734648 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab071b712deb3518b2ecc312d28f3334.slice/crio-26cdc733878b0aa9f6723473017c22a4c8dfcbee40d9a1fb914fd6796cbe5c88 WatchSource:0}: Error finding container 26cdc733878b0aa9f6723473017c22a4c8dfcbee40d9a1fb914fd6796cbe5c88: Status 404 returned error can't find the container with id 26cdc733878b0aa9f6723473017c22a4c8dfcbee40d9a1fb914fd6796cbe5c88 Jun 03 13:47:23.736646 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:23.736624 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed4570c8dc1824eee4a39cfb0b987e1.slice/crio-9fa3576322ac630654468945f61fd3c9afaa3e92795faf4232dc98e3354e4d36 WatchSource:0}: Error finding container 9fa3576322ac630654468945f61fd3c9afaa3e92795faf4232dc98e3354e4d36: Status 404 returned error can't find the container with id 9fa3576322ac630654468945f61fd3c9afaa3e92795faf4232dc98e3354e4d36 Jun 03 13:47:23.737979 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.737963 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 13:47:23.825690 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.825656 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:23.871784 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:23.871760 2573 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:47:23.925811 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:23.925776 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:24.026266 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.026191 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" Jun 03 13:47:24.030644 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.030628 2573 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:47:24.033404 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.033389 2573 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" Jun 03 13:47:24.045558 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.045539 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 13:47:24.046415 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.046403 2573 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" Jun 03 13:47:24.055472 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.055458 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 13:47:24.609879 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.609840 2573 apiserver.go:52] "Watching apiserver" Jun 03 13:47:24.616157 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.616125 2573 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 13:47:24.618855 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.618830 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-m8ltt","openshift-dns/node-resolver-dqxqv","openshift-image-registry/node-ca-9ghdz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal","openshift-multus/network-metrics-daemon-9nbjd","openshift-network-operator/iptables-alerter-rnw2s","openshift-ovn-kubernetes/ovnkube-node-bznnp","kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb","openshift-cluster-node-tuning-operator/tuned-d68j4","openshift-multus/multus-additional-cni-plugins-s44x2","openshift-multus/multus-x7rlv","openshift-network-diagnostics/network-check-target-l4mjx"] Jun 03 13:47:24.621561 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.621541 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.622587 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.622565 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.623589 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.623571 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.624055 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.624038 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 13:47:24.624274 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.624257 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 13:47:24.624398 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.624376 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p5lq9\"" Jun 03 13:47:24.624691 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.624672 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:24.624788 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.624731 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:24.624788 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.624744 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.625246 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.625228 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mvdbx\"" Jun 03 13:47:24.625349 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.625235 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 13:47:24.625349 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.625329 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 13:47:24.625676 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.625657 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 13:47:24.625676 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.625671 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 13:47:24.625823 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.625750 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 13:47:24.625869 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.625856 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cxgq9\"" Jun 03 13:47:24.626874 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.626728 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 13:47:24.626874 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.626846 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.627051 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.627024 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qrht7\"" Jun 03 13:47:24.627309 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.627153 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 13:47:24.627409 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.627327 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 13:47:24.628160 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.628140 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.628970 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.628951 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 13:47:24.628970 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.628939 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 13:47:24.629198 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.629173 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 13:47:24.629457 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.629442 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:24.629711 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.629690 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9gdsm\"" Jun 03 13:47:24.630555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.630533 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 13:47:24.630635 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.630601 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 13:47:24.630725 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.630710 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 13:47:24.632364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.630873 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 13:47:24.632364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.631131 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.632364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.632212 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 13:47:24.632364 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.632225 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 13:47:24.632595 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.632477 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 13:47:24.632595 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.632549 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 13:47:24.633158 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.633091 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fgkx6\"" Jun 03 13:47:24.633813 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.633270 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bf27v\"" Jun 03 13:47:24.633813 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.633285 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.633813 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.633677 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 13:47:24.634027 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.633954 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 13:47:24.634336 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.634310 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 13:47:24.634937 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.634747 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lx7pw\"" Jun 03 13:47:24.634937 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.634769 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 13:47:24.634937 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.634819 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:24.634937 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.634912 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:24.635426 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.635398 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 13:47:24.636477 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.636429 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 13:47:24.636560 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.636430 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bpr6h\"" Jun 03 13:47:24.641375 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641355 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/753361c0-5f5d-4c40-80f6-1a62881ac29a-host-slash\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.641469 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641390 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-system-cni-dir\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.641469 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.641469 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641450 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c78e1418-cf8d-4609-a4a8-f12b567f096d-host\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.641610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641476 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.641610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641493 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-cnibin\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.641610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.641610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641532 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-os-release\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.641610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641555 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65989819-76cd-4637-933a-13fb65e50c43-etc-tuned\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.641610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641572 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-hostroot\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.641610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641601 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-multus-certs\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641623 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qgb2\" (UniqueName: \"kubernetes.io/projected/42ca3328-c9ec-44cc-8ec5-966ce538afce-kube-api-access-4qgb2\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641638 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-run-netns\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641653 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641670 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-cni-multus\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641694 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-ovn\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysctl-conf\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641802 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-socket-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641827 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-env-overrides\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641851 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-kubernetes\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641876 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-sys\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9120bfb5-d3f1-4385-aa41-d563cc5697a0-agent-certs\") pod \"konnectivity-agent-m8ltt\" (UID: \"9120bfb5-d3f1-4385-aa41-d563cc5697a0\") " pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641936 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641958 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-run\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.641985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.641985 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5bfz\" (UniqueName: \"kubernetes.io/projected/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-kube-api-access-x5bfz\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642022 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642047 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-cni-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642073 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-netns\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642109 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-etc-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642157 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysconfig\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642185 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-os-release\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-kubelet\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642283 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-systemd\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642330 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-log-socket\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642441 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrjc8\" (UniqueName: \"kubernetes.io/projected/c78e1418-cf8d-4609-a4a8-f12b567f096d-kube-api-access-lrjc8\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642470 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-system-cni-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642495 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42ca3328-c9ec-44cc-8ec5-966ce538afce-cni-binary-copy\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642534 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-cni-bin\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.642556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642564 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-conf-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642591 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovnkube-config\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642617 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-cnibin\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8pjt\" (UniqueName: \"kubernetes.io/projected/ee004359-f795-4e63-85b4-9a4a3f0765ed-kube-api-access-v8pjt\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642670 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-registration-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642694 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-device-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642736 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5zzq\" (UniqueName: \"kubernetes.io/projected/91f6533e-96a4-4c11-93fe-daf419fea4cf-kube-api-access-g5zzq\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642768 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642794 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovn-node-metrics-cert\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642818 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642848 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-systemd\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642871 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-var-lib-kubelet\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642930 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd87p\" (UniqueName: \"kubernetes.io/projected/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-kube-api-access-wd87p\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642966 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/753361c0-5f5d-4c40-80f6-1a62881ac29a-iptables-alerter-script\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.642994 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-kubelet\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.643202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643039 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-lib-modules\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643230 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-host\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643261 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-sys-fs\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9120bfb5-d3f1-4385-aa41-d563cc5697a0-konnectivity-ca\") pod \"konnectivity-agent-m8ltt\" (UID: \"9120bfb5-d3f1-4385-aa41-d563cc5697a0\") " pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643313 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-k8s-cni-cncf-io\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643343 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clmzj\" (UniqueName: \"kubernetes.io/projected/753361c0-5f5d-4c40-80f6-1a62881ac29a-kube-api-access-clmzj\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643368 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-slash\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643407 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-node-log\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643445 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-modprobe-d\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643479 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghhr6\" (UniqueName: \"kubernetes.io/projected/65989819-76cd-4637-933a-13fb65e50c43-kube-api-access-ghhr6\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643506 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-etc-kubernetes\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643529 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-hosts-file\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643555 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-tmp-dir\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643578 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlnhr\" (UniqueName: \"kubernetes.io/projected/31fb2514-a18b-41df-b762-2c97c0da1b6c-kube-api-access-zlnhr\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643598 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c78e1418-cf8d-4609-a4a8-f12b567f096d-serviceca\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643636 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-socket-dir-parent\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-systemd-units\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.643996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-var-lib-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.644790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovnkube-script-lib\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.644790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.644790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643764 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65989819-76cd-4637-933a-13fb65e50c43-tmp\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.644790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643791 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.644790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643875 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-cni-bin\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.644790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643948 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-cni-netd\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.644790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysctl-d\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.644790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.643995 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-daemon-config\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.672801 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.672768 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 13:42:23 +0000 UTC" deadline="2028-02-28 17:53:40.361411148 +0000 UTC" Jun 03 13:47:24.672801 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.672801 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15244h6m15.688614257s" Jun 03 13:47:24.734248 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.734220 2573 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 13:47:24.737823 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.737770 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" event={"ID":"8ed4570c8dc1824eee4a39cfb0b987e1","Type":"ContainerStarted","Data":"9fa3576322ac630654468945f61fd3c9afaa3e92795faf4232dc98e3354e4d36"} Jun 03 13:47:24.738863 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.738835 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" event={"ID":"ab071b712deb3518b2ecc312d28f3334","Type":"ContainerStarted","Data":"26cdc733878b0aa9f6723473017c22a4c8dfcbee40d9a1fb914fd6796cbe5c88"} Jun 03 13:47:24.744309 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.744289 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42ca3328-c9ec-44cc-8ec5-966ce538afce-cni-binary-copy\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.744420 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.744316 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-cni-bin\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.744420 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.744352 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-conf-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.744420 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.744371 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovnkube-config\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.744420 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.744389 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-cnibin\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.744612 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.744438 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v8pjt\" (UniqueName: \"kubernetes.io/projected/ee004359-f795-4e63-85b4-9a4a3f0765ed-kube-api-access-v8pjt\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.744612 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.744496 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-registration-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.744612 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.744550 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-device-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.745282 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745261 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-cni-bin\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.745386 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745323 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5zzq\" (UniqueName: \"kubernetes.io/projected/91f6533e-96a4-4c11-93fe-daf419fea4cf-kube-api-access-g5zzq\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.745446 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745387 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.745492 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745439 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-conf-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.745492 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745448 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovn-node-metrics-cert\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.745572 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745530 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:24.745572 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745436 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-cnibin\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.745650 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745575 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42ca3328-c9ec-44cc-8ec5-966ce538afce-cni-binary-copy\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.745650 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745630 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-registration-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.745744 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745725 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-device-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.745850 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745806 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.746026 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.745931 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 13:47:24.746084 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746037 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-systemd\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746133 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746082 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-var-lib-kubelet\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746133 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746123 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wd87p\" (UniqueName: \"kubernetes.io/projected/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-kube-api-access-wd87p\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.746230 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746161 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/753361c0-5f5d-4c40-80f6-1a62881ac29a-iptables-alerter-script\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.746230 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746191 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-kubelet\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.746230 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746221 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-lib-modules\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746363 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746252 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-host\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746363 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746282 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-sys-fs\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.746363 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746314 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9120bfb5-d3f1-4385-aa41-d563cc5697a0-konnectivity-ca\") pod \"konnectivity-agent-m8ltt\" (UID: \"9120bfb5-d3f1-4385-aa41-d563cc5697a0\") " pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:24.746363 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746344 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-k8s-cni-cncf-io\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.746542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746379 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clmzj\" (UniqueName: \"kubernetes.io/projected/753361c0-5f5d-4c40-80f6-1a62881ac29a-kube-api-access-clmzj\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.746542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746397 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-kubelet\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.746542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746411 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-slash\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.746542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746429 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-host\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746443 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-node-log\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.746542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746475 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-modprobe-d\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746502 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ghhr6\" (UniqueName: \"kubernetes.io/projected/65989819-76cd-4637-933a-13fb65e50c43-kube-api-access-ghhr6\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746532 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-etc-kubernetes\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746560 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-hosts-file\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746589 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-tmp-dir\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746618 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zlnhr\" (UniqueName: \"kubernetes.io/projected/31fb2514-a18b-41df-b762-2c97c0da1b6c-kube-api-access-zlnhr\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746631 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-slash\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746647 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c78e1418-cf8d-4609-a4a8-f12b567f096d-serviceca\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746660 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-lib-modules\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746675 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-socket-dir-parent\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746705 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-sys-fs\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746754 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-socket-dir-parent\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746756 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-var-lib-kubelet\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746662 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-systemd\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746816 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-node-log\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746858 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-k8s-cni-cncf-io\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.746968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.746933 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-etc-kubernetes\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747074 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-hosts-file\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747199 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-tmp-dir\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747334 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-modprobe-d\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747379 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-systemd-units\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747412 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-var-lib-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747443 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovnkube-script-lib\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747475 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747509 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-var-lib-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747499 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65989819-76cd-4637-933a-13fb65e50c43-tmp\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.747575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747563 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747584 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c78e1418-cf8d-4609-a4a8-f12b567f096d-serviceca\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747594 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-cni-bin\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747630 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-cni-netd\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747667 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysctl-d\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747701 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-daemon-config\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747733 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/753361c0-5f5d-4c40-80f6-1a62881ac29a-host-slash\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747764 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-system-cni-dir\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747783 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9120bfb5-d3f1-4385-aa41-d563cc5697a0-konnectivity-ca\") pod \"konnectivity-agent-m8ltt\" (UID: \"9120bfb5-d3f1-4385-aa41-d563cc5697a0\") " pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747801 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747836 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c78e1418-cf8d-4609-a4a8-f12b567f096d-host\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747872 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747932 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-cnibin\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.747965 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.748053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748005 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-os-release\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748046 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65989819-76cd-4637-933a-13fb65e50c43-etc-tuned\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748110 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-hostroot\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748145 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-multus-certs\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748173 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4qgb2\" (UniqueName: \"kubernetes.io/projected/42ca3328-c9ec-44cc-8ec5-966ce538afce-kube-api-access-4qgb2\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748208 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-run-netns\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748241 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748278 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-cni-multus\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748309 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-ovn\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748342 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysctl-conf\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748374 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-socket-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748406 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-env-overrides\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748433 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-kubernetes\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748466 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-sys\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748483 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovnkube-script-lib\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748543 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748597 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-os-release\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.748695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748641 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-cni-bin\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748662 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748505 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9120bfb5-d3f1-4385-aa41-d563cc5697a0-agent-certs\") pod \"konnectivity-agent-m8ltt\" (UID: \"9120bfb5-d3f1-4385-aa41-d563cc5697a0\") " pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748685 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-cni-netd\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748771 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748794 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysctl-d\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748806 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-run\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748840 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x5bfz\" (UniqueName: \"kubernetes.io/projected/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-kube-api-access-x5bfz\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748954 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-cni-multus\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749042 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-hostroot\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.748599 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-systemd-units\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749118 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-multus-certs\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749148 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749333 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749387 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749422 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-cni-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.749470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749453 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-netns\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749478 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-etc-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749509 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-host-run-netns\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749515 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysconfig\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749563 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysconfig\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749613 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c78e1418-cf8d-4609-a4a8-f12b567f096d-host\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749597 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-os-release\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749650 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-kubelet\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749679 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-systemd\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749670 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-os-release\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749711 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-log-socket\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749744 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749790 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lrjc8\" (UniqueName: \"kubernetes.io/projected/c78e1418-cf8d-4609-a4a8-f12b567f096d-kube-api-access-lrjc8\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749823 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-system-cni-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.749948 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-system-cni-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750012 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750026 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65989819-76cd-4637-933a-13fb65e50c43-tmp\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750121 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-var-lib-kubelet\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.750252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750121 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-etc-openvswitch\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750151 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/753361c0-5f5d-4c40-80f6-1a62881ac29a-host-slash\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750198 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovnkube-config\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750210 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-systemd\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750250 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-run\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.750270 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.750380 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs podName:2c98fc98-abf1-4502-9f2d-cd8bcd1968db nodeName:}" failed. No retries permitted until 2026-06-03 13:47:25.250346352 +0000 UTC m=+3.021035072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs") pod "network-metrics-daemon-9nbjd" (UID: "2c98fc98-abf1-4502-9f2d-cd8bcd1968db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750445 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750566 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91f6533e-96a4-4c11-93fe-daf419fea4cf-socket-dir\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750601 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-kubernetes\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750625 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-run-ovn\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750676 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-system-cni-dir\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750699 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/753361c0-5f5d-4c40-80f6-1a62881ac29a-iptables-alerter-script\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750722 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31fb2514-a18b-41df-b762-2c97c0da1b6c-log-socket\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.751032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750859 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-etc-sysctl-conf\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.751652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.751159 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-daemon-config\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.751652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.751227 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31fb2514-a18b-41df-b762-2c97c0da1b6c-env-overrides\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.751652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.751362 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee004359-f795-4e63-85b4-9a4a3f0765ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.751652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.750269 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-cnibin\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.751652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.751447 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-multus-cni-dir\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.751652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.751496 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42ca3328-c9ec-44cc-8ec5-966ce538afce-host-run-netns\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.751652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.751566 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65989819-76cd-4637-933a-13fb65e50c43-sys\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.751652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.751631 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee004359-f795-4e63-85b4-9a4a3f0765ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.752998 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.752783 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31fb2514-a18b-41df-b762-2c97c0da1b6c-ovn-node-metrics-cert\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.752998 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.752944 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9120bfb5-d3f1-4385-aa41-d563cc5697a0-agent-certs\") pod \"konnectivity-agent-m8ltt\" (UID: \"9120bfb5-d3f1-4385-aa41-d563cc5697a0\") " pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:24.753712 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.753692 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65989819-76cd-4637-933a-13fb65e50c43-etc-tuned\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.754104 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.754081 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:47:24.754203 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.754109 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:47:24.754203 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.754124 2573 projected.go:194] Error preparing data for projected volume kube-api-access-wdpf9 for pod openshift-network-diagnostics/network-check-target-l4mjx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:24.754308 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:24.754224 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9 podName:40b054a1-6cc3-4c1a-90b8-c780b8358de9 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:25.254208069 +0000 UTC m=+3.024896793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wdpf9" (UniqueName: "kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9") pod "network-check-target-l4mjx" (UID: "40b054a1-6cc3-4c1a-90b8-c780b8358de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:24.756262 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.756240 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8pjt\" (UniqueName: \"kubernetes.io/projected/ee004359-f795-4e63-85b4-9a4a3f0765ed-kube-api-access-v8pjt\") pod \"multus-additional-cni-plugins-s44x2\" (UID: \"ee004359-f795-4e63-85b4-9a4a3f0765ed\") " pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.758684 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.758660 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5zzq\" (UniqueName: \"kubernetes.io/projected/91f6533e-96a4-4c11-93fe-daf419fea4cf-kube-api-access-g5zzq\") pod \"aws-ebs-csi-driver-node-54bcb\" (UID: \"91f6533e-96a4-4c11-93fe-daf419fea4cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.763235 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.763188 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qgb2\" (UniqueName: \"kubernetes.io/projected/42ca3328-c9ec-44cc-8ec5-966ce538afce-kube-api-access-4qgb2\") pod \"multus-x7rlv\" (UID: \"42ca3328-c9ec-44cc-8ec5-966ce538afce\") " pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.763338 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.763271 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd87p\" (UniqueName: \"kubernetes.io/projected/8c8e8e92-11f2-4735-a6bc-32c4bc13f517-kube-api-access-wd87p\") pod \"node-resolver-dqxqv\" (UID: \"8c8e8e92-11f2-4735-a6bc-32c4bc13f517\") " pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.764190 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.764162 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5bfz\" (UniqueName: \"kubernetes.io/projected/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-kube-api-access-x5bfz\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:24.764530 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.764509 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghhr6\" (UniqueName: \"kubernetes.io/projected/65989819-76cd-4637-933a-13fb65e50c43-kube-api-access-ghhr6\") pod \"tuned-d68j4\" (UID: \"65989819-76cd-4637-933a-13fb65e50c43\") " pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.764617 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.764597 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlnhr\" (UniqueName: \"kubernetes.io/projected/31fb2514-a18b-41df-b762-2c97c0da1b6c-kube-api-access-zlnhr\") pod \"ovnkube-node-bznnp\" (UID: \"31fb2514-a18b-41df-b762-2c97c0da1b6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.764962 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.764935 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clmzj\" (UniqueName: \"kubernetes.io/projected/753361c0-5f5d-4c40-80f6-1a62881ac29a-kube-api-access-clmzj\") pod \"iptables-alerter-rnw2s\" (UID: \"753361c0-5f5d-4c40-80f6-1a62881ac29a\") " pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.765648 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.765624 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrjc8\" (UniqueName: \"kubernetes.io/projected/c78e1418-cf8d-4609-a4a8-f12b567f096d-kube-api-access-lrjc8\") pod \"node-ca-9ghdz\" (UID: \"c78e1418-cf8d-4609-a4a8-f12b567f096d\") " pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.888562 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.888368 2573 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:47:24.939707 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.939682 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d68j4" Jun 03 13:47:24.946312 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:24.946264 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65989819_76cd_4637_933a_13fb65e50c43.slice/crio-ca53f177004c0dcf8e14b060a110207fc7fb78c4664fdd828f850f5bb26f251a WatchSource:0}: Error finding container ca53f177004c0dcf8e14b060a110207fc7fb78c4664fdd828f850f5bb26f251a: Status 404 returned error can't find the container with id ca53f177004c0dcf8e14b060a110207fc7fb78c4664fdd828f850f5bb26f251a Jun 03 13:47:24.946583 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.946566 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dqxqv" Jun 03 13:47:24.954124 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.954107 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9ghdz" Jun 03 13:47:24.960725 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.960706 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rnw2s" Jun 03 13:47:24.960837 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:24.960794 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78e1418_cf8d_4609_a4a8_f12b567f096d.slice/crio-7c40484aa8e904deb0ecb2f99a20405576d61d21e18398a1854e77a1bb59dd6a WatchSource:0}: Error finding container 7c40484aa8e904deb0ecb2f99a20405576d61d21e18398a1854e77a1bb59dd6a: Status 404 returned error can't find the container with id 7c40484aa8e904deb0ecb2f99a20405576d61d21e18398a1854e77a1bb59dd6a Jun 03 13:47:24.966787 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.966766 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:24.973606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.973291 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" Jun 03 13:47:24.973606 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:24.973568 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fb2514_a18b_41df_b762_2c97c0da1b6c.slice/crio-f8eadb6e541d7823e8cfa9d69e92d0e9587948c5d67b74063bc05689e9437a6a WatchSource:0}: Error finding container f8eadb6e541d7823e8cfa9d69e92d0e9587948c5d67b74063bc05689e9437a6a: Status 404 returned error can't find the container with id f8eadb6e541d7823e8cfa9d69e92d0e9587948c5d67b74063bc05689e9437a6a Jun 03 13:47:24.979437 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.979416 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:24.982792 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:24.982765 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f6533e_96a4_4c11_93fe_daf419fea4cf.slice/crio-a7fd8786d02f947fd8003ad7a7fac48646396c4be71b1de52ce6dafad3950e0a WatchSource:0}: Error finding container a7fd8786d02f947fd8003ad7a7fac48646396c4be71b1de52ce6dafad3950e0a: Status 404 returned error can't find the container with id a7fd8786d02f947fd8003ad7a7fac48646396c4be71b1de52ce6dafad3950e0a Jun 03 13:47:24.985846 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.985827 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s44x2" Jun 03 13:47:24.988228 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:24.988200 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9120bfb5_d3f1_4385_aa41_d563cc5697a0.slice/crio-cf0b12226cf41cfa7320c3249ce42efac7d791c4dfa2b30c97e3fa97d1931961 WatchSource:0}: Error finding container cf0b12226cf41cfa7320c3249ce42efac7d791c4dfa2b30c97e3fa97d1931961: Status 404 returned error can't find the container with id cf0b12226cf41cfa7320c3249ce42efac7d791c4dfa2b30c97e3fa97d1931961 Jun 03 13:47:24.991449 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:24.991428 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x7rlv" Jun 03 13:47:24.994361 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:24.994297 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee004359_f795_4e63_85b4_9a4a3f0765ed.slice/crio-48e6c20673e3c96412fb262097f0c9d909e15877b85400b5abb84e091491a6ca WatchSource:0}: Error finding container 48e6c20673e3c96412fb262097f0c9d909e15877b85400b5abb84e091491a6ca: Status 404 returned error can't find the container with id 48e6c20673e3c96412fb262097f0c9d909e15877b85400b5abb84e091491a6ca Jun 03 13:47:24.999938 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:47:24.999864 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42ca3328_c9ec_44cc_8ec5_966ce538afce.slice/crio-608c9a12c693f70cafdfb19aa1859bd8d4819e055480e31e595f788f3020223c WatchSource:0}: Error finding container 608c9a12c693f70cafdfb19aa1859bd8d4819e055480e31e595f788f3020223c: Status 404 returned error can't find the container with id 608c9a12c693f70cafdfb19aa1859bd8d4819e055480e31e595f788f3020223c Jun 03 13:47:25.129606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.129579 2573 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:47:25.256125 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.256088 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:25.256281 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.256159 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:25.256281 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:25.256268 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:25.256281 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:25.256273 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:47:25.256397 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:25.256295 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:47:25.256397 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:25.256305 2573 projected.go:194] Error preparing data for projected volume kube-api-access-wdpf9 for pod openshift-network-diagnostics/network-check-target-l4mjx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:25.256397 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:25.256330 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs podName:2c98fc98-abf1-4502-9f2d-cd8bcd1968db nodeName:}" failed. No retries permitted until 2026-06-03 13:47:26.256312249 +0000 UTC m=+4.027000958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs") pod "network-metrics-daemon-9nbjd" (UID: "2c98fc98-abf1-4502-9f2d-cd8bcd1968db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:25.256397 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:25.256354 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9 podName:40b054a1-6cc3-4c1a-90b8-c780b8358de9 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:26.256337368 +0000 UTC m=+4.027026072 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdpf9" (UniqueName: "kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9") pod "network-check-target-l4mjx" (UID: "40b054a1-6cc3-4c1a-90b8-c780b8358de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:25.673981 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.673943 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 13:42:23 +0000 UTC" deadline="2028-02-03 16:07:09.365999759 +0000 UTC" Jun 03 13:47:25.673981 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.673978 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14642h19m43.692024338s" Jun 03 13:47:25.734512 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.734478 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:25.734689 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:25.734638 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:25.743740 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.743709 2573 generic.go:358] "Generic (PLEG): container finished" podID="8ed4570c8dc1824eee4a39cfb0b987e1" containerID="0635eea2609fd2347c4a12f81720903146e5c0fca8d3648771e00e2cc76b208b" exitCode=0 Jun 03 13:47:25.743872 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.743782 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" event={"ID":"8ed4570c8dc1824eee4a39cfb0b987e1","Type":"ContainerDied","Data":"0635eea2609fd2347c4a12f81720903146e5c0fca8d3648771e00e2cc76b208b"} Jun 03 13:47:25.758405 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.758373 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x7rlv" event={"ID":"42ca3328-c9ec-44cc-8ec5-966ce538afce","Type":"ContainerStarted","Data":"608c9a12c693f70cafdfb19aa1859bd8d4819e055480e31e595f788f3020223c"} Jun 03 13:47:25.760371 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.759953 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s44x2" event={"ID":"ee004359-f795-4e63-85b4-9a4a3f0765ed","Type":"ContainerStarted","Data":"48e6c20673e3c96412fb262097f0c9d909e15877b85400b5abb84e091491a6ca"} Jun 03 13:47:25.774063 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.774036 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m8ltt" event={"ID":"9120bfb5-d3f1-4385-aa41-d563cc5697a0","Type":"ContainerStarted","Data":"cf0b12226cf41cfa7320c3249ce42efac7d791c4dfa2b30c97e3fa97d1931961"} Jun 03 13:47:25.784098 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.784070 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"f8eadb6e541d7823e8cfa9d69e92d0e9587948c5d67b74063bc05689e9437a6a"} Jun 03 13:47:25.786853 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.786827 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9ghdz" event={"ID":"c78e1418-cf8d-4609-a4a8-f12b567f096d","Type":"ContainerStarted","Data":"7c40484aa8e904deb0ecb2f99a20405576d61d21e18398a1854e77a1bb59dd6a"} Jun 03 13:47:25.795804 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.795771 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dqxqv" event={"ID":"8c8e8e92-11f2-4735-a6bc-32c4bc13f517","Type":"ContainerStarted","Data":"cb0e00ed7fe579c2ca6897ad1c8448f232b6351311b66972524d65f6e52b60a6"} Jun 03 13:47:25.798546 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.798521 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" event={"ID":"91f6533e-96a4-4c11-93fe-daf419fea4cf","Type":"ContainerStarted","Data":"a7fd8786d02f947fd8003ad7a7fac48646396c4be71b1de52ce6dafad3950e0a"} Jun 03 13:47:25.801458 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.801434 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rnw2s" event={"ID":"753361c0-5f5d-4c40-80f6-1a62881ac29a","Type":"ContainerStarted","Data":"99b85ae0db24078a17e0c2c9daa3567e6717b4558666bc650ea770624a8de629"} Jun 03 13:47:25.808546 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:25.808515 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d68j4" event={"ID":"65989819-76cd-4637-933a-13fb65e50c43","Type":"ContainerStarted","Data":"ca53f177004c0dcf8e14b060a110207fc7fb78c4664fdd828f850f5bb26f251a"} Jun 03 13:47:26.264405 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:26.264356 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:26.264595 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:26.264462 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:26.264656 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:26.264605 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:26.264706 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:26.264672 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs podName:2c98fc98-abf1-4502-9f2d-cd8bcd1968db nodeName:}" failed. No retries permitted until 2026-06-03 13:47:28.264651358 +0000 UTC m=+6.035340079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs") pod "network-metrics-daemon-9nbjd" (UID: "2c98fc98-abf1-4502-9f2d-cd8bcd1968db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:26.265145 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:26.265122 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:47:26.265250 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:26.265149 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:47:26.265250 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:26.265163 2573 projected.go:194] Error preparing data for projected volume kube-api-access-wdpf9 for pod openshift-network-diagnostics/network-check-target-l4mjx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:26.265250 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:26.265211 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9 podName:40b054a1-6cc3-4c1a-90b8-c780b8358de9 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:28.265195698 +0000 UTC m=+6.035884403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdpf9" (UniqueName: "kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9") pod "network-check-target-l4mjx" (UID: "40b054a1-6cc3-4c1a-90b8-c780b8358de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:26.736566 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:26.734128 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:26.736566 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:26.734256 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:27.734628 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:27.734098 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:27.734628 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:27.734248 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:28.280559 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:28.280484 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:28.281040 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:28.280604 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:28.281040 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:28.280663 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:47:28.281040 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:28.280706 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:47:28.281040 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:28.280720 2573 projected.go:194] Error preparing data for projected volume kube-api-access-wdpf9 for pod openshift-network-diagnostics/network-check-target-l4mjx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:28.281040 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:28.280743 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:28.281040 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:28.280774 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9 podName:40b054a1-6cc3-4c1a-90b8-c780b8358de9 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:32.280755166 +0000 UTC m=+10.051443874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdpf9" (UniqueName: "kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9") pod "network-check-target-l4mjx" (UID: "40b054a1-6cc3-4c1a-90b8-c780b8358de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:28.281040 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:28.280795 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs podName:2c98fc98-abf1-4502-9f2d-cd8bcd1968db nodeName:}" failed. No retries permitted until 2026-06-03 13:47:32.28078518 +0000 UTC m=+10.051473889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs") pod "network-metrics-daemon-9nbjd" (UID: "2c98fc98-abf1-4502-9f2d-cd8bcd1968db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:28.736844 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:28.736331 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:28.736844 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:28.736450 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:29.734129 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:29.734098 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:29.734596 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:29.734245 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:30.734128 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:30.734092 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:30.734312 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:30.734208 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:31.734043 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:31.734007 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:31.734214 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:31.734147 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:32.313589 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:32.313544 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:32.314083 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:32.313616 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:32.314083 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:32.313787 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:47:32.314083 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:32.313808 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:47:32.314083 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:32.313820 2573 projected.go:194] Error preparing data for projected volume kube-api-access-wdpf9 for pod openshift-network-diagnostics/network-check-target-l4mjx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:32.314083 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:32.313899 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9 podName:40b054a1-6cc3-4c1a-90b8-c780b8358de9 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:40.313863356 +0000 UTC m=+18.084552064 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdpf9" (UniqueName: "kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9") pod "network-check-target-l4mjx" (UID: "40b054a1-6cc3-4c1a-90b8-c780b8358de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:32.314083 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:32.314063 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:32.314351 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:32.314134 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs podName:2c98fc98-abf1-4502-9f2d-cd8bcd1968db nodeName:}" failed. No retries permitted until 2026-06-03 13:47:40.314115346 +0000 UTC m=+18.084804054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs") pod "network-metrics-daemon-9nbjd" (UID: "2c98fc98-abf1-4502-9f2d-cd8bcd1968db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:32.734472 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:32.734417 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:32.734611 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:32.734526 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:33.733755 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:33.733725 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:33.734215 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:33.733858 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:34.733754 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:34.733723 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:34.734260 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:34.733840 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:35.733534 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:35.733498 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:35.733716 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:35.733630 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:36.734154 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:36.734128 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:36.734549 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:36.734224 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:37.733870 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:37.733829 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:37.734036 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:37.733971 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:38.263253 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.263219 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-j4b9r"] Jun 03 13:47:38.266384 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.266359 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.266510 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:38.266441 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:38.359185 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.359147 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.359316 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.359199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-dbus\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.359316 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.359288 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-kubelet-config\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.460528 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.460490 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.460528 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.460538 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-dbus\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.460749 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.460614 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-kubelet-config\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.460749 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:38.460647 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:38.460749 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.460706 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-kubelet-config\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.460749 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:38.460722 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret podName:a7566a4f-1ff5-45ae-93fe-7c91e3223f28 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:38.960701857 +0000 UTC m=+16.731390575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret") pod "global-pull-secret-syncer-j4b9r" (UID: "a7566a4f-1ff5-45ae-93fe-7c91e3223f28") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:38.460982 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.460762 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-dbus\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.733854 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.733811 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:38.734044 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:38.733960 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:38.963277 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:38.963236 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:38.963445 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:38.963408 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:38.963492 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:38.963479 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret podName:a7566a4f-1ff5-45ae-93fe-7c91e3223f28 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:39.963458252 +0000 UTC m=+17.734146969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret") pod "global-pull-secret-syncer-j4b9r" (UID: "a7566a4f-1ff5-45ae-93fe-7c91e3223f28") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:39.734270 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:39.734229 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:39.734748 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:39.734229 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:39.734748 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:39.734356 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:39.734748 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:39.734476 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:39.972397 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:39.972359 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:39.972568 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:39.972528 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:39.972623 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:39.972596 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret podName:a7566a4f-1ff5-45ae-93fe-7c91e3223f28 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:41.972577973 +0000 UTC m=+19.743266684 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret") pod "global-pull-secret-syncer-j4b9r" (UID: "a7566a4f-1ff5-45ae-93fe-7c91e3223f28") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:40.374929 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:40.374878 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:40.375122 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:40.374945 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:40.375122 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:40.375009 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:40.375122 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:40.375088 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs podName:2c98fc98-abf1-4502-9f2d-cd8bcd1968db nodeName:}" failed. No retries permitted until 2026-06-03 13:47:56.375071429 +0000 UTC m=+34.145760132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs") pod "network-metrics-daemon-9nbjd" (UID: "2c98fc98-abf1-4502-9f2d-cd8bcd1968db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:40.375122 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:40.375096 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:47:40.375122 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:40.375113 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:47:40.375122 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:40.375124 2573 projected.go:194] Error preparing data for projected volume kube-api-access-wdpf9 for pod openshift-network-diagnostics/network-check-target-l4mjx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:40.375429 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:40.375174 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9 podName:40b054a1-6cc3-4c1a-90b8-c780b8358de9 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:56.375159799 +0000 UTC m=+34.145848502 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdpf9" (UniqueName: "kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9") pod "network-check-target-l4mjx" (UID: "40b054a1-6cc3-4c1a-90b8-c780b8358de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:40.733579 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:40.733545 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:40.733758 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:40.733671 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:41.733569 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:41.733537 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:41.733976 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:41.733543 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:41.733976 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:41.733637 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:41.733976 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:41.733749 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:41.988057 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:41.987967 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:41.988195 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:41.988146 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:41.988246 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:41.988223 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret podName:a7566a4f-1ff5-45ae-93fe-7c91e3223f28 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:45.988208107 +0000 UTC m=+23.758896815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret") pod "global-pull-secret-syncer-j4b9r" (UID: "a7566a4f-1ff5-45ae-93fe-7c91e3223f28") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:42.734693 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:42.734668 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:42.735006 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:42.734739 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:43.734050 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.733677 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:43.734474 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:43.734152 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:43.734474 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.733694 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:43.734474 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:43.734292 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:43.840370 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.840336 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" event={"ID":"8ed4570c8dc1824eee4a39cfb0b987e1","Type":"ContainerStarted","Data":"9b70c3cd6a439fe10ea83a110b5542b32e1da0d26ef25b1e162ec541dfe00add"} Jun 03 13:47:43.842072 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.842030 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" event={"ID":"ab071b712deb3518b2ecc312d28f3334","Type":"ContainerStarted","Data":"f82c47144ec6847dad28433ca33a2ffaa29b211ef2bcef2be8771e38b8a77d4e"} Jun 03 13:47:43.843575 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.843550 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x7rlv" event={"ID":"42ca3328-c9ec-44cc-8ec5-966ce538afce","Type":"ContainerStarted","Data":"70d4a0908320dd48bff44582bda5c33b9eedb5a2ded9f7836f102839035e9b1c"} Jun 03 13:47:43.845177 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.845149 2573 generic.go:358] "Generic (PLEG): container finished" podID="ee004359-f795-4e63-85b4-9a4a3f0765ed" containerID="2a1685af40d9281353b47a5baf857e58daafce3b8de9cccd44edd5a3fbb69ec0" exitCode=0 Jun 03 13:47:43.845290 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.845221 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s44x2" event={"ID":"ee004359-f795-4e63-85b4-9a4a3f0765ed","Type":"ContainerDied","Data":"2a1685af40d9281353b47a5baf857e58daafce3b8de9cccd44edd5a3fbb69ec0"} Jun 03 13:47:43.847377 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.847127 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m8ltt" event={"ID":"9120bfb5-d3f1-4385-aa41-d563cc5697a0","Type":"ContainerStarted","Data":"85cc0813ab1cfd46dd04ecfaffb10880f5c4e0400bad63bc54c072143ac890c5"} Jun 03 13:47:43.850093 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.850073 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 13:47:43.850406 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.850338 2573 generic.go:358] "Generic (PLEG): container finished" podID="31fb2514-a18b-41df-b762-2c97c0da1b6c" containerID="86dd2fefef2e913b6d99133c9f4203192ed1f9d10e167648d98a4c4a458e6a40" exitCode=1 Jun 03 13:47:43.850406 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.850367 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"2b522fa8b5bb2889b7187e751864f0c89b4aa7e03c6ab05619e103f23a3d63b7"} Jun 03 13:47:43.850406 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.850396 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"63e6d84e2eb39e6d8292c34986010244a8e33a32f2152f0f784e9a365660e4cd"} Jun 03 13:47:43.850585 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.850423 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"795d532feeb1befb59e2068e9b738260312b6efea59faab139a8580745eb2006"} Jun 03 13:47:43.850585 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.850437 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"92daeb960e84fdcac0ae7c4b0df97577120a90a73786afa99a48093d018bf3cb"} Jun 03 13:47:43.850585 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.850449 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerDied","Data":"86dd2fefef2e913b6d99133c9f4203192ed1f9d10e167648d98a4c4a458e6a40"} Jun 03 13:47:43.850585 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.850464 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"35dcbbe1b89db82e67dcd6b54eea46ca2232a86690b49e0d75810716c7cd64c3"} Jun 03 13:47:43.851973 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.851950 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9ghdz" event={"ID":"c78e1418-cf8d-4609-a4a8-f12b567f096d","Type":"ContainerStarted","Data":"87e3957fa9595f6a4212abd493f6f5e6661f5e136e37a862bfe24811e18022bc"} Jun 03 13:47:43.853394 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.853364 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dqxqv" event={"ID":"8c8e8e92-11f2-4735-a6bc-32c4bc13f517","Type":"ContainerStarted","Data":"5a451d28685909436f82d103b144a0c0fecb763583b8624de4c92cf928f475c3"} Jun 03 13:47:43.854646 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.854628 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" event={"ID":"91f6533e-96a4-4c11-93fe-daf419fea4cf","Type":"ContainerStarted","Data":"ca79d24d777eb08ad1e848f21f6b8fe7483877568a8b94953c5dcccc6af681fd"} Jun 03 13:47:43.855661 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.855644 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d68j4" event={"ID":"65989819-76cd-4637-933a-13fb65e50c43","Type":"ContainerStarted","Data":"597a30e228e6903875dbbba05558cf2741caf53965e97cfbc2381baabbfe6355"} Jun 03 13:47:43.880220 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.880174 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" podStartSLOduration=19.880158547 podStartE2EDuration="19.880158547s" podCreationTimestamp="2026-06-03 13:47:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:47:43.860328803 +0000 UTC m=+21.631017529" watchObservedRunningTime="2026-06-03 13:47:43.880158547 +0000 UTC m=+21.650847271" Jun 03 13:47:43.880454 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.880437 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" podStartSLOduration=19.880431692 podStartE2EDuration="19.880431692s" podCreationTimestamp="2026-06-03 13:47:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:47:43.880075146 +0000 UTC m=+21.650763885" watchObservedRunningTime="2026-06-03 13:47:43.880431692 +0000 UTC m=+21.651120417" Jun 03 13:47:43.903010 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.902963 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d68j4" podStartSLOduration=3.855973693 podStartE2EDuration="21.902945092s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:24.949421425 +0000 UTC m=+2.720110142" lastFinishedPulling="2026-06-03 13:47:42.996392826 +0000 UTC m=+20.767081541" observedRunningTime="2026-06-03 13:47:43.90226277 +0000 UTC m=+21.672951496" watchObservedRunningTime="2026-06-03 13:47:43.902945092 +0000 UTC m=+21.673633820" Jun 03 13:47:43.924401 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.924348 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-dqxqv" podStartSLOduration=3.906296608 podStartE2EDuration="21.924321814s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:24.954177635 +0000 UTC m=+2.724866341" lastFinishedPulling="2026-06-03 13:47:42.972202834 +0000 UTC m=+20.742891547" observedRunningTime="2026-06-03 13:47:43.924212284 +0000 UTC m=+21.694901011" watchObservedRunningTime="2026-06-03 13:47:43.924321814 +0000 UTC m=+21.695010542" Jun 03 13:47:43.951121 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.951053 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9ghdz" podStartSLOduration=3.921441592 podStartE2EDuration="21.951032006s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:24.962450094 +0000 UTC m=+2.733138797" lastFinishedPulling="2026-06-03 13:47:42.992040501 +0000 UTC m=+20.762729211" observedRunningTime="2026-06-03 13:47:43.946987766 +0000 UTC m=+21.717676493" watchObservedRunningTime="2026-06-03 13:47:43.951032006 +0000 UTC m=+21.721720732" Jun 03 13:47:43.969616 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:43.969564 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m8ltt" podStartSLOduration=4.297351509 podStartE2EDuration="21.969547814s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:24.990858367 +0000 UTC m=+2.761547075" lastFinishedPulling="2026-06-03 13:47:42.663054673 +0000 UTC m=+20.433743380" observedRunningTime="2026-06-03 13:47:43.969292994 +0000 UTC m=+21.739981719" watchObservedRunningTime="2026-06-03 13:47:43.969547814 +0000 UTC m=+21.740236545" Jun 03 13:47:44.036211 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:44.036159 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-x7rlv" podStartSLOduration=4.004843564 podStartE2EDuration="22.036143056s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:25.001781016 +0000 UTC m=+2.772469722" lastFinishedPulling="2026-06-03 13:47:43.033080511 +0000 UTC m=+20.803769214" observedRunningTime="2026-06-03 13:47:44.035518127 +0000 UTC m=+21.806206853" watchObservedRunningTime="2026-06-03 13:47:44.036143056 +0000 UTC m=+21.806831781" Jun 03 13:47:44.507740 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:44.507671 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 13:47:44.707118 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:44.706972 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T13:47:44.507694875Z","UUID":"4ebdc1b8-df90-4ee7-bba8-343ed3df7a02","Handler":null,"Name":"","Endpoint":""} Jun 03 13:47:44.710596 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:44.710572 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 13:47:44.710596 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:44.710603 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 13:47:44.733569 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:44.733544 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:44.733710 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:44.733642 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:44.859718 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:44.859631 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" event={"ID":"91f6533e-96a4-4c11-93fe-daf419fea4cf","Type":"ContainerStarted","Data":"c685c765af66633195265cd6e9dd1dde22555bc1f79a379a5380b2aa678bad8e"} Jun 03 13:47:44.861278 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:44.861242 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rnw2s" event={"ID":"753361c0-5f5d-4c40-80f6-1a62881ac29a","Type":"ContainerStarted","Data":"502cced07dfc65bd3f017e3642fffba2c47b1294bfc03b2b528408ea6b2a3d67"} Jun 03 13:47:45.733840 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:45.733811 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:45.733985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:45.733816 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:45.733985 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:45.733957 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:45.734101 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:45.734023 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:45.865127 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:45.864878 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" event={"ID":"91f6533e-96a4-4c11-93fe-daf419fea4cf","Type":"ContainerStarted","Data":"48c4e77a4a8345360801af08f66c658ca1946c5f47947c695ab4ec2852557934"} Jun 03 13:47:45.887413 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:45.887350 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rnw2s" podStartSLOduration=5.886336475 podStartE2EDuration="23.887336005s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:24.971125394 +0000 UTC m=+2.741814098" lastFinishedPulling="2026-06-03 13:47:42.972124924 +0000 UTC m=+20.742813628" observedRunningTime="2026-06-03 13:47:44.887017454 +0000 UTC m=+22.657706179" watchObservedRunningTime="2026-06-03 13:47:45.887336005 +0000 UTC m=+23.658024713" Jun 03 13:47:45.887686 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:45.887655 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54bcb" podStartSLOduration=3.2293568759999998 podStartE2EDuration="23.887647267s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:24.985500624 +0000 UTC m=+2.756189330" lastFinishedPulling="2026-06-03 13:47:45.643790999 +0000 UTC m=+23.414479721" observedRunningTime="2026-06-03 13:47:45.887537371 +0000 UTC m=+23.658226095" watchObservedRunningTime="2026-06-03 13:47:45.887647267 +0000 UTC m=+23.658335991" Jun 03 13:47:45.918849 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:45.918811 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:45.919521 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:45.919498 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:46.022659 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:46.022098 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:46.022659 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:46.022244 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:46.022659 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:46.022320 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret podName:a7566a4f-1ff5-45ae-93fe-7c91e3223f28 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:54.022299413 +0000 UTC m=+31.792988129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret") pod "global-pull-secret-syncer-j4b9r" (UID: "a7566a4f-1ff5-45ae-93fe-7c91e3223f28") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:46.734308 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:46.734271 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:46.734529 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:46.734408 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:46.869592 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:46.869563 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 13:47:46.870146 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:46.869986 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"f454a5a83bc3046a7ba23a0ecbc4ecb87b4c70775b74dfc1118581fc12a1f9a9"} Jun 03 13:47:46.870449 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:46.870426 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:46.870761 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:46.870744 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m8ltt" Jun 03 13:47:47.733849 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:47.733806 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:47.734053 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:47.733806 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:47.734053 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:47.733953 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:47.734053 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:47.734026 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:48.734147 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.733974 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:48.734733 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:48.734212 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:48.874966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.874931 2573 generic.go:358] "Generic (PLEG): container finished" podID="ee004359-f795-4e63-85b4-9a4a3f0765ed" containerID="ebae12cb8c0eb08132cecffcff952b891f12abbe24a40fd4bc4ba90cb31ac553" exitCode=0 Jun 03 13:47:48.875109 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.875010 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s44x2" event={"ID":"ee004359-f795-4e63-85b4-9a4a3f0765ed","Type":"ContainerDied","Data":"ebae12cb8c0eb08132cecffcff952b891f12abbe24a40fd4bc4ba90cb31ac553"} Jun 03 13:47:48.877813 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.877797 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 13:47:48.878189 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.878167 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"55d34c493bcdd6561d8fc17e6047f2046e3f3e7f5ddb3641c2c67e59e79f143c"} Jun 03 13:47:48.878597 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.878576 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:48.878685 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.878603 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:48.878856 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.878731 2573 scope.go:117] "RemoveContainer" containerID="86dd2fefef2e913b6d99133c9f4203192ed1f9d10e167648d98a4c4a458e6a40" Jun 03 13:47:48.895559 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:48.895532 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:49.734276 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:49.734244 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:49.734627 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:49.734349 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:49.734627 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:49.734402 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:49.734627 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:49.734487 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:49.883790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:49.883763 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 13:47:49.884173 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:49.884146 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" event={"ID":"31fb2514-a18b-41df-b762-2c97c0da1b6c","Type":"ContainerStarted","Data":"4811ad18547a2ff1edb527fd70bcddc42925301f4616001b01900f16b6fd7c86"} Jun 03 13:47:49.884507 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:49.884481 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:49.899799 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:49.899777 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:47:49.969447 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:49.969408 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" podStartSLOduration=9.892179722 podStartE2EDuration="27.969393734s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:24.979319169 +0000 UTC m=+2.750007876" lastFinishedPulling="2026-06-03 13:47:43.056533186 +0000 UTC m=+20.827221888" observedRunningTime="2026-06-03 13:47:49.92506402 +0000 UTC m=+27.695752746" watchObservedRunningTime="2026-06-03 13:47:49.969393734 +0000 UTC m=+27.740082457" Jun 03 13:47:50.265386 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:50.265170 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j4b9r"] Jun 03 13:47:50.265510 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:50.265407 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:50.265510 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:50.265495 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:50.268611 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:50.268587 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9nbjd"] Jun 03 13:47:50.268730 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:50.268684 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:50.268783 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:50.268761 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:50.281866 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:50.281840 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l4mjx"] Jun 03 13:47:50.281988 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:50.281977 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:50.282072 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:50.282054 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:50.887931 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:50.887871 2573 generic.go:358] "Generic (PLEG): container finished" podID="ee004359-f795-4e63-85b4-9a4a3f0765ed" containerID="667a67206fdd55c206cfc802207ef69e3a13ea1670842346d5417d1eeed85f39" exitCode=0 Jun 03 13:47:50.888311 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:50.887952 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s44x2" event={"ID":"ee004359-f795-4e63-85b4-9a4a3f0765ed","Type":"ContainerDied","Data":"667a67206fdd55c206cfc802207ef69e3a13ea1670842346d5417d1eeed85f39"} Jun 03 13:47:51.734223 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:51.734183 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:51.734425 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:51.734191 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:51.734425 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:51.734320 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:51.734425 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:51.734384 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:52.734531 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:52.734505 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:52.735066 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:52.734593 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:52.894008 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:52.893976 2573 generic.go:358] "Generic (PLEG): container finished" podID="ee004359-f795-4e63-85b4-9a4a3f0765ed" containerID="ac4a1f0f5d46002be598db9ca7b2c692714176d3707bc9518b71669c3626be97" exitCode=0 Jun 03 13:47:52.894133 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:52.894036 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s44x2" event={"ID":"ee004359-f795-4e63-85b4-9a4a3f0765ed","Type":"ContainerDied","Data":"ac4a1f0f5d46002be598db9ca7b2c692714176d3707bc9518b71669c3626be97"} Jun 03 13:47:53.733762 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:53.733724 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:53.733945 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:53.733730 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:53.733945 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:53.733842 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:53.734048 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:53.733970 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:54.082515 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:54.082372 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:54.083156 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:54.082534 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:54.083156 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:54.082614 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret podName:a7566a4f-1ff5-45ae-93fe-7c91e3223f28 nodeName:}" failed. No retries permitted until 2026-06-03 13:48:10.082593211 +0000 UTC m=+47.853281914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret") pod "global-pull-secret-syncer-j4b9r" (UID: "a7566a4f-1ff5-45ae-93fe-7c91e3223f28") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:47:54.734234 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:54.734198 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:54.734428 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:54.734319 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:55.733772 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:55.733736 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:55.734224 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:55.733865 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbjd" podUID="2c98fc98-abf1-4502-9f2d-cd8bcd1968db" Jun 03 13:47:55.734224 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:55.733920 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:55.734224 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:55.734005 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l4mjx" podUID="40b054a1-6cc3-4c1a-90b8-c780b8358de9" Jun 03 13:47:56.396336 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:56.396301 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:56.396574 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:56.396362 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:56.396574 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:56.396440 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:56.396574 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:56.396502 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:47:56.396574 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:56.396520 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:47:56.396574 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:56.396533 2573 projected.go:194] Error preparing data for projected volume kube-api-access-wdpf9 for pod openshift-network-diagnostics/network-check-target-l4mjx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:56.396574 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:56.396521 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs podName:2c98fc98-abf1-4502-9f2d-cd8bcd1968db nodeName:}" failed. No retries permitted until 2026-06-03 13:48:28.396497762 +0000 UTC m=+66.167186465 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs") pod "network-metrics-daemon-9nbjd" (UID: "2c98fc98-abf1-4502-9f2d-cd8bcd1968db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:47:56.396839 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:56.396592 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9 podName:40b054a1-6cc3-4c1a-90b8-c780b8358de9 nodeName:}" failed. No retries permitted until 2026-06-03 13:48:28.396574927 +0000 UTC m=+66.167263637 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdpf9" (UniqueName: "kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9") pod "network-check-target-l4mjx" (UID: "40b054a1-6cc3-4c1a-90b8-c780b8358de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:47:56.734359 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:56.734327 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:56.734791 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:56.734448 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j4b9r" podUID="a7566a4f-1ff5-45ae-93fe-7c91e3223f28" Jun 03 13:47:57.033060 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.032992 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeReady" Jun 03 13:47:57.033202 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.033145 2573 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 13:47:57.095200 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.095164 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7ptv9"] Jun 03 13:47:57.128251 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.128224 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77b75f88c5-7smnt"] Jun 03 13:47:57.128429 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.128390 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:47:57.132632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.132599 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 13:47:57.132779 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.132651 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 13:47:57.132779 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.132651 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-68pcr\"" Jun 03 13:47:57.133655 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.133632 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 13:47:57.142827 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.142805 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cmq8g"] Jun 03 13:47:57.142987 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.142966 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.146925 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.146907 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 03 13:47:57.147108 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.147087 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 03 13:47:57.147388 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.147371 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c7qvm\"" Jun 03 13:47:57.147574 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.147556 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 03 13:47:57.153925 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.153879 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 03 13:47:57.162941 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.162920 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7ptv9"] Jun 03 13:47:57.163045 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.162948 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cmq8g"] Jun 03 13:47:57.163045 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.162961 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77b75f88c5-7smnt"] Jun 03 13:47:57.163147 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.163079 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.166112 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.166096 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 13:47:57.166353 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.166334 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4dx2f\"" Jun 03 13:47:57.166542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.166526 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 13:47:57.304921 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.304825 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.304921 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.304866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-ca-trust-extracted\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.305131 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.304928 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-tmp-dir\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.305131 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.304952 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9krf\" (UniqueName: \"kubernetes.io/projected/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-kube-api-access-t9krf\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.305131 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.304972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-trusted-ca\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.305131 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.304995 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-bound-sa-token\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.305131 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.305050 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:47:57.305131 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.305085 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8ht7\" (UniqueName: \"kubernetes.io/projected/72f705bc-93cc-4b36-bd92-ef3ad046561e-kube-api-access-z8ht7\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:47:57.305131 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.305102 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-image-registry-private-configuration\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.305413 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.305150 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.305413 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.305175 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-certificates\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.305413 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.305233 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-config-volume\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.305413 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.305261 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-installation-pull-secrets\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.305413 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.305301 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfqsz\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-kube-api-access-qfqsz\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.406234 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406196 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqsz\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-kube-api-access-qfqsz\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.406234 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406242 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.406441 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406270 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-ca-trust-extracted\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.406441 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.406383 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:47:57.406509 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.406449 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls podName:7ba04155-ffdc-4d87-be26-adc7e6e7bbd6 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:57.906431031 +0000 UTC m=+35.677119734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls") pod "dns-default-cmq8g" (UID: "7ba04155-ffdc-4d87-be26-adc7e6e7bbd6") : secret "dns-default-metrics-tls" not found Jun 03 13:47:57.406582 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406557 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-tmp-dir\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.406722 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406596 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t9krf\" (UniqueName: \"kubernetes.io/projected/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-kube-api-access-t9krf\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.406722 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406617 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-trusted-ca\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.406722 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406632 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-bound-sa-token\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.406722 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406669 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:47:57.406722 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406683 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-ca-trust-extracted\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.406722 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406694 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8ht7\" (UniqueName: \"kubernetes.io/projected/72f705bc-93cc-4b36-bd92-ef3ad046561e-kube-api-access-z8ht7\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:47:57.407065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406737 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-image-registry-private-configuration\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.407065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406773 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.407065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406800 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-certificates\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.407065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406841 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-config-volume\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.407065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.406899 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-installation-pull-secrets\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.407303 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.407260 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-tmp-dir\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.407360 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.407319 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:47:57.407409 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.407386 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert podName:72f705bc-93cc-4b36-bd92-ef3ad046561e nodeName:}" failed. No retries permitted until 2026-06-03 13:47:57.90736845 +0000 UTC m=+35.678057168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert") pod "ingress-canary-7ptv9" (UID: "72f705bc-93cc-4b36-bd92-ef3ad046561e") : secret "canary-serving-cert" not found Jun 03 13:47:57.407480 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.407465 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:47:57.407533 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.407484 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b75f88c5-7smnt: secret "image-registry-tls" not found Jun 03 13:47:57.407533 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.407524 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls podName:46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b nodeName:}" failed. No retries permitted until 2026-06-03 13:47:57.907509311 +0000 UTC m=+35.678198018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls") pod "image-registry-77b75f88c5-7smnt" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b") : secret "image-registry-tls" not found Jun 03 13:47:57.408014 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.407699 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-certificates\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.408014 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.407736 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-trusted-ca\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.408014 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.407772 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-config-volume\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.412323 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.412302 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-installation-pull-secrets\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.412424 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.412305 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-image-registry-private-configuration\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.415757 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.415730 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-bound-sa-token\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.416271 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.416249 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8ht7\" (UniqueName: \"kubernetes.io/projected/72f705bc-93cc-4b36-bd92-ef3ad046561e-kube-api-access-z8ht7\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:47:57.416405 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.416391 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9krf\" (UniqueName: \"kubernetes.io/projected/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-kube-api-access-t9krf\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.416552 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.416525 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfqsz\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-kube-api-access-qfqsz\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.734585 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.734551 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:47:57.735071 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.734551 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:47:57.738330 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.738306 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 13:47:57.738472 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.738306 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 13:47:57.738624 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.738607 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 13:47:57.739145 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.739125 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z5xc9\"" Jun 03 13:47:57.739344 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.739322 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hmwfp\"" Jun 03 13:47:57.911175 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.911142 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:47:57.911175 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.911179 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:57.911420 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:57.911253 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:57.911420 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.911318 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:47:57.911420 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.911335 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:47:57.911420 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.911361 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:47:57.911420 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.911380 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b75f88c5-7smnt: secret "image-registry-tls" not found Jun 03 13:47:57.911420 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.911399 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert podName:72f705bc-93cc-4b36-bd92-ef3ad046561e nodeName:}" failed. No retries permitted until 2026-06-03 13:47:58.911376416 +0000 UTC m=+36.682065153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert") pod "ingress-canary-7ptv9" (UID: "72f705bc-93cc-4b36-bd92-ef3ad046561e") : secret "canary-serving-cert" not found Jun 03 13:47:57.911420 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.911416 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls podName:7ba04155-ffdc-4d87-be26-adc7e6e7bbd6 nodeName:}" failed. No retries permitted until 2026-06-03 13:47:58.911409051 +0000 UTC m=+36.682097754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls") pod "dns-default-cmq8g" (UID: "7ba04155-ffdc-4d87-be26-adc7e6e7bbd6") : secret "dns-default-metrics-tls" not found Jun 03 13:47:57.911420 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:57.911426 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls podName:46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b nodeName:}" failed. No retries permitted until 2026-06-03 13:47:58.911421524 +0000 UTC m=+36.682110227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls") pod "image-registry-77b75f88c5-7smnt" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b") : secret "image-registry-tls" not found Jun 03 13:47:58.734357 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:58.734323 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:47:58.737144 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:58.737118 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 13:47:58.920669 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:58.920632 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:47:58.920844 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:58.920711 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:47:58.920844 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:58.920744 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:47:58.921001 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:58.920868 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:47:58.921001 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:58.920948 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert podName:72f705bc-93cc-4b36-bd92-ef3ad046561e nodeName:}" failed. No retries permitted until 2026-06-03 13:48:00.92092942 +0000 UTC m=+38.691618140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert") pod "ingress-canary-7ptv9" (UID: "72f705bc-93cc-4b36-bd92-ef3ad046561e") : secret "canary-serving-cert" not found Jun 03 13:47:58.921001 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:58.920948 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:47:58.921001 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:58.920968 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b75f88c5-7smnt: secret "image-registry-tls" not found Jun 03 13:47:58.921203 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:58.921022 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls podName:46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b nodeName:}" failed. No retries permitted until 2026-06-03 13:48:00.921005141 +0000 UTC m=+38.691693858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls") pod "image-registry-77b75f88c5-7smnt" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b") : secret "image-registry-tls" not found Jun 03 13:47:58.921203 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:58.920872 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:47:58.921203 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:47:58.921062 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls podName:7ba04155-ffdc-4d87-be26-adc7e6e7bbd6 nodeName:}" failed. No retries permitted until 2026-06-03 13:48:00.921052387 +0000 UTC m=+38.691741092 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls") pod "dns-default-cmq8g" (UID: "7ba04155-ffdc-4d87-be26-adc7e6e7bbd6") : secret "dns-default-metrics-tls" not found Jun 03 13:47:59.912827 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:59.912799 2573 generic.go:358] "Generic (PLEG): container finished" podID="ee004359-f795-4e63-85b4-9a4a3f0765ed" containerID="8ed6b21773d023d85589d876db79e6e26dc39c852d76f9a816530e3f0b1d5442" exitCode=0 Jun 03 13:47:59.913233 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:47:59.912849 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s44x2" event={"ID":"ee004359-f795-4e63-85b4-9a4a3f0765ed","Type":"ContainerDied","Data":"8ed6b21773d023d85589d876db79e6e26dc39c852d76f9a816530e3f0b1d5442"} Jun 03 13:48:00.917487 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:00.917458 2573 generic.go:358] "Generic (PLEG): container finished" podID="ee004359-f795-4e63-85b4-9a4a3f0765ed" containerID="fff71e1b4ef1b6457f91df09d7c7e0101f17f7f30275d82e849e7e09ae64261b" exitCode=0 Jun 03 13:48:00.917998 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:00.917516 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s44x2" event={"ID":"ee004359-f795-4e63-85b4-9a4a3f0765ed","Type":"ContainerDied","Data":"fff71e1b4ef1b6457f91df09d7c7e0101f17f7f30275d82e849e7e09ae64261b"} Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:00.937201 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:00.937274 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:00.937303 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:00.937433 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:00.937446 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b75f88c5-7smnt: secret "image-registry-tls" not found Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:00.937500 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls podName:46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b nodeName:}" failed. No retries permitted until 2026-06-03 13:48:04.937481755 +0000 UTC m=+42.708170461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls") pod "image-registry-77b75f88c5-7smnt" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b") : secret "image-registry-tls" not found Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:00.937898 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:00.937942 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls podName:7ba04155-ffdc-4d87-be26-adc7e6e7bbd6 nodeName:}" failed. No retries permitted until 2026-06-03 13:48:04.937928869 +0000 UTC m=+42.708617573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls") pod "dns-default-cmq8g" (UID: "7ba04155-ffdc-4d87-be26-adc7e6e7bbd6") : secret "dns-default-metrics-tls" not found Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:00.937995 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:48:00.948909 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:00.938023 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert podName:72f705bc-93cc-4b36-bd92-ef3ad046561e nodeName:}" failed. No retries permitted until 2026-06-03 13:48:04.938011894 +0000 UTC m=+42.708700603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert") pod "ingress-canary-7ptv9" (UID: "72f705bc-93cc-4b36-bd92-ef3ad046561e") : secret "canary-serving-cert" not found Jun 03 13:48:01.921929 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:01.921896 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s44x2" event={"ID":"ee004359-f795-4e63-85b4-9a4a3f0765ed","Type":"ContainerStarted","Data":"70b48a0bf321de319b8aa754ab9b784d60418a63a1084ba1948259f7141ea120"} Jun 03 13:48:01.950255 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:01.950200 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s44x2" podStartSLOduration=6.086057292 podStartE2EDuration="39.95018526s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:47:24.996349005 +0000 UTC m=+2.767037712" lastFinishedPulling="2026-06-03 13:47:58.860476977 +0000 UTC m=+36.631165680" observedRunningTime="2026-06-03 13:48:01.949699847 +0000 UTC m=+39.720388572" watchObservedRunningTime="2026-06-03 13:48:01.95018526 +0000 UTC m=+39.720873987" Jun 03 13:48:02.706875 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.706842 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl"] Jun 03 13:48:02.708775 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.708759 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" Jun 03 13:48:02.714912 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.714875 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 03 13:48:02.715470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.715455 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 03 13:48:02.715470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.715464 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mqws4\"" Jun 03 13:48:02.737205 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.737176 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl"] Jun 03 13:48:02.850589 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.850546 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkv4\" (UniqueName: \"kubernetes.io/projected/5af435e1-70e4-4200-86c4-3fccc88e9f0f-kube-api-access-vpkv4\") pod \"migrator-96c6f749-lrfnl\" (UID: \"5af435e1-70e4-4200-86c4-3fccc88e9f0f\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" Jun 03 13:48:02.951623 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.951592 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkv4\" (UniqueName: \"kubernetes.io/projected/5af435e1-70e4-4200-86c4-3fccc88e9f0f-kube-api-access-vpkv4\") pod \"migrator-96c6f749-lrfnl\" (UID: \"5af435e1-70e4-4200-86c4-3fccc88e9f0f\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" Jun 03 13:48:02.960965 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:02.960908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkv4\" (UniqueName: \"kubernetes.io/projected/5af435e1-70e4-4200-86c4-3fccc88e9f0f-kube-api-access-vpkv4\") pod \"migrator-96c6f749-lrfnl\" (UID: \"5af435e1-70e4-4200-86c4-3fccc88e9f0f\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" Jun 03 13:48:03.016769 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:03.016746 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" Jun 03 13:48:03.154227 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:03.154194 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl"] Jun 03 13:48:03.159512 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:03.159482 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af435e1_70e4_4200_86c4_3fccc88e9f0f.slice/crio-11fd56891f0c0871e4385a3971c2e94cc422cf5ff0077f2ec0759177a383a261 WatchSource:0}: Error finding container 11fd56891f0c0871e4385a3971c2e94cc422cf5ff0077f2ec0759177a383a261: Status 404 returned error can't find the container with id 11fd56891f0c0871e4385a3971c2e94cc422cf5ff0077f2ec0759177a383a261 Jun 03 13:48:03.287546 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:03.287465 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dqxqv_8c8e8e92-11f2-4735-a6bc-32c4bc13f517/dns-node-resolver/0.log" Jun 03 13:48:03.927155 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:03.927120 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" event={"ID":"5af435e1-70e4-4200-86c4-3fccc88e9f0f","Type":"ContainerStarted","Data":"11fd56891f0c0871e4385a3971c2e94cc422cf5ff0077f2ec0759177a383a261"} Jun 03 13:48:04.258390 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.258316 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-hm8ls"] Jun 03 13:48:04.260174 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.260158 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.263174 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.263149 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rvcpt\"" Jun 03 13:48:04.263476 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.263457 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 03 13:48:04.263562 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.263474 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 03 13:48:04.264293 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.264275 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 03 13:48:04.264386 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.264311 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 03 13:48:04.270810 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.270791 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-hm8ls"] Jun 03 13:48:04.295994 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.295975 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9ghdz_c78e1418-cf8d-4609-a4a8-f12b567f096d/node-ca/0.log" Jun 03 13:48:04.363223 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.363184 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce20f597-fa23-4805-8a4b-e96a1ead2edd-signing-key\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.363372 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.363246 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmcwj\" (UniqueName: \"kubernetes.io/projected/ce20f597-fa23-4805-8a4b-e96a1ead2edd-kube-api-access-gmcwj\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.363372 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.363286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce20f597-fa23-4805-8a4b-e96a1ead2edd-signing-cabundle\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.463723 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.463691 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce20f597-fa23-4805-8a4b-e96a1ead2edd-signing-key\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.463878 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.463737 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gmcwj\" (UniqueName: \"kubernetes.io/projected/ce20f597-fa23-4805-8a4b-e96a1ead2edd-kube-api-access-gmcwj\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.463878 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.463764 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce20f597-fa23-4805-8a4b-e96a1ead2edd-signing-cabundle\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.464395 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.464368 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce20f597-fa23-4805-8a4b-e96a1ead2edd-signing-cabundle\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.466153 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.466134 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce20f597-fa23-4805-8a4b-e96a1ead2edd-signing-key\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.472869 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.472848 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmcwj\" (UniqueName: \"kubernetes.io/projected/ce20f597-fa23-4805-8a4b-e96a1ead2edd-kube-api-access-gmcwj\") pod \"service-ca-685df94d57-hm8ls\" (UID: \"ce20f597-fa23-4805-8a4b-e96a1ead2edd\") " pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.569107 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.569011 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-hm8ls" Jun 03 13:48:04.693826 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.693794 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-hm8ls"] Jun 03 13:48:04.696738 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:04.696699 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce20f597_fa23_4805_8a4b_e96a1ead2edd.slice/crio-c7f571666ce01527f51f476e19e35c32f3f7dda014fe94ed7f3f05ae78dcf386 WatchSource:0}: Error finding container c7f571666ce01527f51f476e19e35c32f3f7dda014fe94ed7f3f05ae78dcf386: Status 404 returned error can't find the container with id c7f571666ce01527f51f476e19e35c32f3f7dda014fe94ed7f3f05ae78dcf386 Jun 03 13:48:04.930395 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.930359 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-hm8ls" event={"ID":"ce20f597-fa23-4805-8a4b-e96a1ead2edd","Type":"ContainerStarted","Data":"c7f571666ce01527f51f476e19e35c32f3f7dda014fe94ed7f3f05ae78dcf386"} Jun 03 13:48:04.968839 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.968790 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:48:04.969023 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.968930 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:48:04.969091 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:04.969034 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:48:04.969091 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:04.968941 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:48:04.969198 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:04.969135 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert podName:72f705bc-93cc-4b36-bd92-ef3ad046561e nodeName:}" failed. No retries permitted until 2026-06-03 13:48:12.969117599 +0000 UTC m=+50.739806305 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert") pod "ingress-canary-7ptv9" (UID: "72f705bc-93cc-4b36-bd92-ef3ad046561e") : secret "canary-serving-cert" not found Jun 03 13:48:04.969198 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:04.969168 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:48:04.969307 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:04.969230 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls podName:7ba04155-ffdc-4d87-be26-adc7e6e7bbd6 nodeName:}" failed. No retries permitted until 2026-06-03 13:48:12.969211838 +0000 UTC m=+50.739900546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls") pod "dns-default-cmq8g" (UID: "7ba04155-ffdc-4d87-be26-adc7e6e7bbd6") : secret "dns-default-metrics-tls" not found Jun 03 13:48:04.969307 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:04.968997 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:48:04.969307 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:04.969250 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77b75f88c5-7smnt: secret "image-registry-tls" not found Jun 03 13:48:04.969307 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:04.969282 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls podName:46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b nodeName:}" failed. No retries permitted until 2026-06-03 13:48:12.96927201 +0000 UTC m=+50.739960714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls") pod "image-registry-77b75f88c5-7smnt" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b") : secret "image-registry-tls" not found Jun 03 13:48:05.934356 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:05.934323 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" event={"ID":"5af435e1-70e4-4200-86c4-3fccc88e9f0f","Type":"ContainerStarted","Data":"c8fcbeb2ba41078365ed41552d8c600a39b8acebe6cd0a57a728cd6b0fe8a83c"} Jun 03 13:48:05.934356 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:05.934360 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" event={"ID":"5af435e1-70e4-4200-86c4-3fccc88e9f0f","Type":"ContainerStarted","Data":"ac0ab28ecc8dc5735675119ac2234e23c0e5ea790640c697d3ec608875fbbba3"} Jun 03 13:48:07.940640 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:07.940600 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-hm8ls" event={"ID":"ce20f597-fa23-4805-8a4b-e96a1ead2edd","Type":"ContainerStarted","Data":"f315a9bec08add975fa248a48ea0f6d10ffa7906496b8b7c349c7c4ce05a633a"} Jun 03 13:48:07.961606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:07.961539 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lrfnl" podStartSLOduration=4.027132743 podStartE2EDuration="5.961524613s" podCreationTimestamp="2026-06-03 13:48:02 +0000 UTC" firstStartedPulling="2026-06-03 13:48:03.161369868 +0000 UTC m=+40.932058570" lastFinishedPulling="2026-06-03 13:48:05.095761723 +0000 UTC m=+42.866450440" observedRunningTime="2026-06-03 13:48:05.961994384 +0000 UTC m=+43.732683108" watchObservedRunningTime="2026-06-03 13:48:07.961524613 +0000 UTC m=+45.732213338" Jun 03 13:48:07.962054 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:07.962020 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-hm8ls" podStartSLOduration=1.616720209 podStartE2EDuration="3.962007199s" podCreationTimestamp="2026-06-03 13:48:04 +0000 UTC" firstStartedPulling="2026-06-03 13:48:04.698521002 +0000 UTC m=+42.469209705" lastFinishedPulling="2026-06-03 13:48:07.043807989 +0000 UTC m=+44.814496695" observedRunningTime="2026-06-03 13:48:07.961601712 +0000 UTC m=+45.732290439" watchObservedRunningTime="2026-06-03 13:48:07.962007199 +0000 UTC m=+45.732695924" Jun 03 13:48:10.109994 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:10.109948 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:48:10.112292 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:10.112271 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7566a4f-1ff5-45ae-93fe-7c91e3223f28-original-pull-secret\") pod \"global-pull-secret-syncer-j4b9r\" (UID: \"a7566a4f-1ff5-45ae-93fe-7c91e3223f28\") " pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:48:10.143232 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:10.143196 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j4b9r" Jun 03 13:48:10.277208 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:10.277176 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7566a4f_1ff5_45ae_93fe_7c91e3223f28.slice/crio-76dfb4d90615e5e63f5c41188cef0116e3eb8385bad7cba13b7a8c3f823fd952 WatchSource:0}: Error finding container 76dfb4d90615e5e63f5c41188cef0116e3eb8385bad7cba13b7a8c3f823fd952: Status 404 returned error can't find the container with id 76dfb4d90615e5e63f5c41188cef0116e3eb8385bad7cba13b7a8c3f823fd952 Jun 03 13:48:10.282314 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:10.282293 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j4b9r"] Jun 03 13:48:10.948039 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:10.947994 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j4b9r" event={"ID":"a7566a4f-1ff5-45ae-93fe-7c91e3223f28","Type":"ContainerStarted","Data":"76dfb4d90615e5e63f5c41188cef0116e3eb8385bad7cba13b7a8c3f823fd952"} Jun 03 13:48:13.037147 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:13.037104 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:48:13.037719 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:13.037171 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:48:13.037719 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:13.037259 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:48:13.037719 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:13.037284 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:48:13.037719 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:48:13.037363 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert podName:72f705bc-93cc-4b36-bd92-ef3ad046561e nodeName:}" failed. No retries permitted until 2026-06-03 13:48:29.037342675 +0000 UTC m=+66.808031384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert") pod "ingress-canary-7ptv9" (UID: "72f705bc-93cc-4b36-bd92-ef3ad046561e") : secret "canary-serving-cert" not found Jun 03 13:48:13.040024 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:13.040001 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ba04155-ffdc-4d87-be26-adc7e6e7bbd6-metrics-tls\") pod \"dns-default-cmq8g\" (UID: \"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6\") " pod="openshift-dns/dns-default-cmq8g" Jun 03 13:48:13.040143 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:13.040065 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"image-registry-77b75f88c5-7smnt\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:48:13.053265 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:13.053157 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:48:13.073023 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:13.072994 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cmq8g" Jun 03 13:48:14.533076 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:14.533043 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cmq8g"] Jun 03 13:48:14.536073 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:14.536018 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba04155_ffdc_4d87_be26_adc7e6e7bbd6.slice/crio-464348af7aef4f899256aefc5c563354e9de5f899a58c33162057fb6fd04b8d3 WatchSource:0}: Error finding container 464348af7aef4f899256aefc5c563354e9de5f899a58c33162057fb6fd04b8d3: Status 404 returned error can't find the container with id 464348af7aef4f899256aefc5c563354e9de5f899a58c33162057fb6fd04b8d3 Jun 03 13:48:14.559218 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:14.559192 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77b75f88c5-7smnt"] Jun 03 13:48:14.725072 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:14.724984 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46f5f71e_0cdf_44ba_a5b8_3b1fb74e494b.slice/crio-32a55e155e0d00738a74ee8991920aafcb63b2a803449952e556751af4d2a705 WatchSource:0}: Error finding container 32a55e155e0d00738a74ee8991920aafcb63b2a803449952e556751af4d2a705: Status 404 returned error can't find the container with id 32a55e155e0d00738a74ee8991920aafcb63b2a803449952e556751af4d2a705 Jun 03 13:48:14.959076 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:14.959044 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j4b9r" event={"ID":"a7566a4f-1ff5-45ae-93fe-7c91e3223f28","Type":"ContainerStarted","Data":"00363b117a1ce5ad2868f7669bd31e989f3539486e8ad0f233ede146796dbf6b"} Jun 03 13:48:14.960312 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:14.960285 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cmq8g" event={"ID":"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6","Type":"ContainerStarted","Data":"464348af7aef4f899256aefc5c563354e9de5f899a58c33162057fb6fd04b8d3"} Jun 03 13:48:14.961680 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:14.961653 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" event={"ID":"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b","Type":"ContainerStarted","Data":"dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c"} Jun 03 13:48:14.961680 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:14.961686 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" event={"ID":"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b","Type":"ContainerStarted","Data":"32a55e155e0d00738a74ee8991920aafcb63b2a803449952e556751af4d2a705"} Jun 03 13:48:14.961844 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:14.961787 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:48:15.005241 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:15.005147 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" podStartSLOduration=40.005132261 podStartE2EDuration="40.005132261s" podCreationTimestamp="2026-06-03 13:47:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:48:15.003870829 +0000 UTC m=+52.774559557" watchObservedRunningTime="2026-06-03 13:48:15.005132261 +0000 UTC m=+52.775820999" Jun 03 13:48:15.005390 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:15.005347 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j4b9r" podStartSLOduration=32.517632486 podStartE2EDuration="37.005342637s" podCreationTimestamp="2026-06-03 13:47:38 +0000 UTC" firstStartedPulling="2026-06-03 13:48:10.279641918 +0000 UTC m=+48.050330639" lastFinishedPulling="2026-06-03 13:48:14.767352086 +0000 UTC m=+52.538040790" observedRunningTime="2026-06-03 13:48:14.979466662 +0000 UTC m=+52.750155388" watchObservedRunningTime="2026-06-03 13:48:15.005342637 +0000 UTC m=+52.776031371" Jun 03 13:48:16.968190 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:16.968156 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cmq8g" event={"ID":"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6","Type":"ContainerStarted","Data":"33ec890c828248fa76d01a7aa27282532813ea279c93f64e8c3c3fb2beffa033"} Jun 03 13:48:16.968190 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:16.968191 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cmq8g" event={"ID":"7ba04155-ffdc-4d87-be26-adc7e6e7bbd6","Type":"ContainerStarted","Data":"df3e226176ec252103988dc3a370bfa1216bee4bd1c1c52f459bcb214c142e87"} Jun 03 13:48:16.968694 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:16.968322 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cmq8g" Jun 03 13:48:16.988895 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:16.988838 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-cmq8g" podStartSLOduration=17.988518675 podStartE2EDuration="19.988821092s" podCreationTimestamp="2026-06-03 13:47:57 +0000 UTC" firstStartedPulling="2026-06-03 13:48:14.538113674 +0000 UTC m=+52.308802380" lastFinishedPulling="2026-06-03 13:48:16.53841609 +0000 UTC m=+54.309104797" observedRunningTime="2026-06-03 13:48:16.987681068 +0000 UTC m=+54.758369794" watchObservedRunningTime="2026-06-03 13:48:16.988821092 +0000 UTC m=+54.759509819" Jun 03 13:48:21.902781 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:21.902746 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bznnp" Jun 03 13:48:26.973768 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:26.973739 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cmq8g" Jun 03 13:48:28.409533 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.409499 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs"] Jun 03 13:48:28.412497 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.412476 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc"] Jun 03 13:48:28.412699 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.412677 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.414629 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.414611 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" Jun 03 13:48:28.415844 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.415814 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 03 13:48:28.416033 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.416011 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 03 13:48:28.416569 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.416554 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 03 13:48:28.416649 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.416570 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 03 13:48:28.417089 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.417074 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 03 13:48:28.417186 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.417171 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8pmrs\"" Jun 03 13:48:28.427250 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.427228 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc"] Jun 03 13:48:28.427953 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.427936 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs"] Jun 03 13:48:28.456836 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.456810 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:48:28.456998 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.456846 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5m99\" (UniqueName: \"kubernetes.io/projected/01f6d122-ba9f-47f6-976c-c71587e64f2c-kube-api-access-g5m99\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.456998 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.456866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/10fb9ad6-8d8a-4944-be96-428dd232c0a6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b6f9844db-x4snc\" (UID: \"10fb9ad6-8d8a-4944-be96-428dd232c0a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" Jun 03 13:48:28.456998 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.456916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g4d7\" (UniqueName: \"kubernetes.io/projected/10fb9ad6-8d8a-4944-be96-428dd232c0a6-kube-api-access-9g4d7\") pod \"managed-serviceaccount-addon-agent-5b6f9844db-x4snc\" (UID: \"10fb9ad6-8d8a-4944-be96-428dd232c0a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" Jun 03 13:48:28.456998 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.456985 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01f6d122-ba9f-47f6-976c-c71587e64f2c-tmp\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.457159 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.457011 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/01f6d122-ba9f-47f6-976c-c71587e64f2c-klusterlet-config\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.457159 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.457061 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:48:28.459506 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.459486 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 13:48:28.459590 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.459573 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 13:48:28.470197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.470173 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c98fc98-abf1-4502-9f2d-cd8bcd1968db-metrics-certs\") pod \"network-metrics-daemon-9nbjd\" (UID: \"2c98fc98-abf1-4502-9f2d-cd8bcd1968db\") " pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:48:28.470343 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.470283 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 13:48:28.480701 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.480672 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdpf9\" (UniqueName: \"kubernetes.io/projected/40b054a1-6cc3-4c1a-90b8-c780b8358de9-kube-api-access-wdpf9\") pod \"network-check-target-l4mjx\" (UID: \"40b054a1-6cc3-4c1a-90b8-c780b8358de9\") " pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:48:28.523757 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.523728 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-vtbnp"] Jun 03 13:48:28.525814 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.525799 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-vtbnp" Jun 03 13:48:28.528646 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.528612 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 03 13:48:28.528901 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.528872 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-pf2w2\"" Jun 03 13:48:28.533307 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.533290 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 03 13:48:28.539556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.539536 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dn9ds"] Jun 03 13:48:28.541585 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.541568 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.542676 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.542653 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-vtbnp"] Jun 03 13:48:28.545096 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.545078 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 13:48:28.545192 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.545079 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kpj7b\"" Jun 03 13:48:28.545392 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.545378 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 13:48:28.545453 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.545405 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 13:48:28.549161 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.549144 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 13:48:28.557333 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.557315 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9g4d7\" (UniqueName: \"kubernetes.io/projected/10fb9ad6-8d8a-4944-be96-428dd232c0a6-kube-api-access-9g4d7\") pod \"managed-serviceaccount-addon-agent-5b6f9844db-x4snc\" (UID: \"10fb9ad6-8d8a-4944-be96-428dd232c0a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" Jun 03 13:48:28.557440 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.557356 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01f6d122-ba9f-47f6-976c-c71587e64f2c-tmp\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.557440 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.557379 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/01f6d122-ba9f-47f6-976c-c71587e64f2c-klusterlet-config\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.557550 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.557449 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5m99\" (UniqueName: \"kubernetes.io/projected/01f6d122-ba9f-47f6-976c-c71587e64f2c-kube-api-access-g5m99\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.557550 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.557498 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/10fb9ad6-8d8a-4944-be96-428dd232c0a6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b6f9844db-x4snc\" (UID: \"10fb9ad6-8d8a-4944-be96-428dd232c0a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" Jun 03 13:48:28.557742 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.557724 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01f6d122-ba9f-47f6-976c-c71587e64f2c-tmp\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.559871 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.559847 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/01f6d122-ba9f-47f6-976c-c71587e64f2c-klusterlet-config\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.559967 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.559936 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/10fb9ad6-8d8a-4944-be96-428dd232c0a6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b6f9844db-x4snc\" (UID: \"10fb9ad6-8d8a-4944-be96-428dd232c0a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" Jun 03 13:48:28.572949 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.572928 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dn9ds"] Jun 03 13:48:28.576808 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.576789 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5m99\" (UniqueName: \"kubernetes.io/projected/01f6d122-ba9f-47f6-976c-c71587e64f2c-kube-api-access-g5m99\") pod \"klusterlet-addon-workmgr-645fb9d878-nqccs\" (UID: \"01f6d122-ba9f-47f6-976c-c71587e64f2c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.578345 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.578312 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g4d7\" (UniqueName: \"kubernetes.io/projected/10fb9ad6-8d8a-4944-be96-428dd232c0a6-kube-api-access-9g4d7\") pod \"managed-serviceaccount-addon-agent-5b6f9844db-x4snc\" (UID: \"10fb9ad6-8d8a-4944-be96-428dd232c0a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" Jun 03 13:48:28.649308 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.649279 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hmwfp\"" Jun 03 13:48:28.653420 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.653403 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z5xc9\"" Jun 03 13:48:28.657517 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.657500 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:48:28.657765 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.657745 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpxdx\" (UniqueName: \"kubernetes.io/projected/4bef7fe1-c7ed-429f-bb84-80b0871b530c-kube-api-access-gpxdx\") pod \"downloads-c65b8dc4b-vtbnp\" (UID: \"4bef7fe1-c7ed-429f-bb84-80b0871b530c\") " pod="openshift-console/downloads-c65b8dc4b-vtbnp" Jun 03 13:48:28.657807 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.657777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zplkl\" (UniqueName: \"kubernetes.io/projected/3042a477-badc-4cb2-9b86-ae27a9649d57-kube-api-access-zplkl\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.657807 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.657799 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3042a477-badc-4cb2-9b86-ae27a9649d57-crio-socket\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.657871 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.657826 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3042a477-badc-4cb2-9b86-ae27a9649d57-data-volume\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.657959 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.657941 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3042a477-badc-4cb2-9b86-ae27a9649d57-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.658010 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.657977 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3042a477-badc-4cb2-9b86-ae27a9649d57-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.662184 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.662150 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbjd" Jun 03 13:48:28.723086 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.723051 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:28.733755 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.733728 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" Jun 03 13:48:28.759289 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.758473 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3042a477-badc-4cb2-9b86-ae27a9649d57-crio-socket\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.759289 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.758524 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3042a477-badc-4cb2-9b86-ae27a9649d57-data-volume\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.759289 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.758583 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3042a477-badc-4cb2-9b86-ae27a9649d57-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.759289 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.758618 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3042a477-badc-4cb2-9b86-ae27a9649d57-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.759289 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.758679 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpxdx\" (UniqueName: \"kubernetes.io/projected/4bef7fe1-c7ed-429f-bb84-80b0871b530c-kube-api-access-gpxdx\") pod \"downloads-c65b8dc4b-vtbnp\" (UID: \"4bef7fe1-c7ed-429f-bb84-80b0871b530c\") " pod="openshift-console/downloads-c65b8dc4b-vtbnp" Jun 03 13:48:28.759289 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.758715 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zplkl\" (UniqueName: \"kubernetes.io/projected/3042a477-badc-4cb2-9b86-ae27a9649d57-kube-api-access-zplkl\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.759289 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.758732 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3042a477-badc-4cb2-9b86-ae27a9649d57-crio-socket\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.759824 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.759804 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3042a477-badc-4cb2-9b86-ae27a9649d57-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.760126 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.760105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3042a477-badc-4cb2-9b86-ae27a9649d57-data-volume\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.762950 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.762902 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3042a477-badc-4cb2-9b86-ae27a9649d57-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.772074 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.772021 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpxdx\" (UniqueName: \"kubernetes.io/projected/4bef7fe1-c7ed-429f-bb84-80b0871b530c-kube-api-access-gpxdx\") pod \"downloads-c65b8dc4b-vtbnp\" (UID: \"4bef7fe1-c7ed-429f-bb84-80b0871b530c\") " pod="openshift-console/downloads-c65b8dc4b-vtbnp" Jun 03 13:48:28.773384 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.773360 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zplkl\" (UniqueName: \"kubernetes.io/projected/3042a477-badc-4cb2-9b86-ae27a9649d57-kube-api-access-zplkl\") pod \"insights-runtime-extractor-dn9ds\" (UID: \"3042a477-badc-4cb2-9b86-ae27a9649d57\") " pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.821539 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.821505 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l4mjx"] Jun 03 13:48:28.825285 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.825261 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9nbjd"] Jun 03 13:48:28.826431 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:28.826401 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b054a1_6cc3_4c1a_90b8_c780b8358de9.slice/crio-f6de7320cc8c04896013a7313c8c0d8dadccda9da94dd76f23ee64cd7da6d403 WatchSource:0}: Error finding container f6de7320cc8c04896013a7313c8c0d8dadccda9da94dd76f23ee64cd7da6d403: Status 404 returned error can't find the container with id f6de7320cc8c04896013a7313c8c0d8dadccda9da94dd76f23ee64cd7da6d403 Jun 03 13:48:28.828031 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:28.828009 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c98fc98_abf1_4502_9f2d_cd8bcd1968db.slice/crio-7f145f39840903da2e10f4f1aa508438cab949344c74eb97d72bd51f73fbd587 WatchSource:0}: Error finding container 7f145f39840903da2e10f4f1aa508438cab949344c74eb97d72bd51f73fbd587: Status 404 returned error can't find the container with id 7f145f39840903da2e10f4f1aa508438cab949344c74eb97d72bd51f73fbd587 Jun 03 13:48:28.834483 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.834457 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-vtbnp" Jun 03 13:48:28.850742 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.850715 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dn9ds" Jun 03 13:48:28.892688 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.892625 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs"] Jun 03 13:48:28.901509 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:28.901089 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc"] Jun 03 13:48:28.929385 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:28.929343 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10fb9ad6_8d8a_4944_be96_428dd232c0a6.slice/crio-527019efeffe1989aecf4d97c6292b205034e2b6eb8c9961f3f8d0ff28666b22 WatchSource:0}: Error finding container 527019efeffe1989aecf4d97c6292b205034e2b6eb8c9961f3f8d0ff28666b22: Status 404 returned error can't find the container with id 527019efeffe1989aecf4d97c6292b205034e2b6eb8c9961f3f8d0ff28666b22 Jun 03 13:48:29.004695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.004658 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" event={"ID":"10fb9ad6-8d8a-4944-be96-428dd232c0a6","Type":"ContainerStarted","Data":"527019efeffe1989aecf4d97c6292b205034e2b6eb8c9961f3f8d0ff28666b22"} Jun 03 13:48:29.006303 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.005930 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9nbjd" event={"ID":"2c98fc98-abf1-4502-9f2d-cd8bcd1968db","Type":"ContainerStarted","Data":"7f145f39840903da2e10f4f1aa508438cab949344c74eb97d72bd51f73fbd587"} Jun 03 13:48:29.007043 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.007021 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l4mjx" event={"ID":"40b054a1-6cc3-4c1a-90b8-c780b8358de9","Type":"ContainerStarted","Data":"f6de7320cc8c04896013a7313c8c0d8dadccda9da94dd76f23ee64cd7da6d403"} Jun 03 13:48:29.007135 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.007089 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-vtbnp"] Jun 03 13:48:29.008009 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.007986 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" event={"ID":"01f6d122-ba9f-47f6-976c-c71587e64f2c","Type":"ContainerStarted","Data":"409da964b4c2bf089737497e918e5a905d4d055d5cc6041698dc510a25c6eeea"} Jun 03 13:48:29.008188 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:29.008170 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bef7fe1_c7ed_429f_bb84_80b0871b530c.slice/crio-c15d17577c0440760a37046eea5b5aaff2dcb09cc69c5d700b14709f20955be7 WatchSource:0}: Error finding container c15d17577c0440760a37046eea5b5aaff2dcb09cc69c5d700b14709f20955be7: Status 404 returned error can't find the container with id c15d17577c0440760a37046eea5b5aaff2dcb09cc69c5d700b14709f20955be7 Jun 03 13:48:29.027594 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.027570 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dn9ds"] Jun 03 13:48:29.028105 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:29.028081 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3042a477_badc_4cb2_9b86_ae27a9649d57.slice/crio-81631b40ab00e201d3f2436d7486a4c6da6a4ab5c999970a07e733da2ec3f637 WatchSource:0}: Error finding container 81631b40ab00e201d3f2436d7486a4c6da6a4ab5c999970a07e733da2ec3f637: Status 404 returned error can't find the container with id 81631b40ab00e201d3f2436d7486a4c6da6a4ab5c999970a07e733da2ec3f637 Jun 03 13:48:29.061234 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.061205 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:48:29.064213 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.064187 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72f705bc-93cc-4b36-bd92-ef3ad046561e-cert\") pod \"ingress-canary-7ptv9\" (UID: \"72f705bc-93cc-4b36-bd92-ef3ad046561e\") " pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:48:29.242733 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.242660 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-68pcr\"" Jun 03 13:48:29.250841 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.250812 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7ptv9" Jun 03 13:48:29.415444 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:29.415412 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7ptv9"] Jun 03 13:48:29.419267 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:29.419236 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f705bc_93cc_4b36_bd92_ef3ad046561e.slice/crio-3e496aa232b9c69b4027622376b855efb6e39deda32f91bd3d8aad65ecafc524 WatchSource:0}: Error finding container 3e496aa232b9c69b4027622376b855efb6e39deda32f91bd3d8aad65ecafc524: Status 404 returned error can't find the container with id 3e496aa232b9c69b4027622376b855efb6e39deda32f91bd3d8aad65ecafc524 Jun 03 13:48:30.016982 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:30.016895 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7ptv9" event={"ID":"72f705bc-93cc-4b36-bd92-ef3ad046561e","Type":"ContainerStarted","Data":"3e496aa232b9c69b4027622376b855efb6e39deda32f91bd3d8aad65ecafc524"} Jun 03 13:48:30.022328 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:30.022293 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dn9ds" event={"ID":"3042a477-badc-4cb2-9b86-ae27a9649d57","Type":"ContainerStarted","Data":"de57142e10d0e5275d5c770812a00d7777598fd5c3e195dac973f14fa8b2f5f5"} Jun 03 13:48:30.022478 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:30.022338 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dn9ds" event={"ID":"3042a477-badc-4cb2-9b86-ae27a9649d57","Type":"ContainerStarted","Data":"6c94849020d135255459ba31305708824471d5a88d9e8df02d8be311d7c13612"} Jun 03 13:48:30.022478 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:30.022356 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dn9ds" event={"ID":"3042a477-badc-4cb2-9b86-ae27a9649d57","Type":"ContainerStarted","Data":"81631b40ab00e201d3f2436d7486a4c6da6a4ab5c999970a07e733da2ec3f637"} Jun 03 13:48:30.024144 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:30.024100 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-vtbnp" event={"ID":"4bef7fe1-c7ed-429f-bb84-80b0871b530c","Type":"ContainerStarted","Data":"c15d17577c0440760a37046eea5b5aaff2dcb09cc69c5d700b14709f20955be7"} Jun 03 13:48:31.041605 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:31.041563 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9nbjd" event={"ID":"2c98fc98-abf1-4502-9f2d-cd8bcd1968db","Type":"ContainerStarted","Data":"0b03626b474e52c3717b16561b0e9794f887ad3bd92baca5eb42e695ab93cf52"} Jun 03 13:48:31.042109 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:31.041610 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9nbjd" event={"ID":"2c98fc98-abf1-4502-9f2d-cd8bcd1968db","Type":"ContainerStarted","Data":"11daf68c626feab01a2815bfb392560861f9fb36701ed260181d3311898002d6"} Jun 03 13:48:32.760228 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:32.760151 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9nbjd" podStartSLOduration=69.414712373 podStartE2EDuration="1m10.760132492s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:48:28.830000961 +0000 UTC m=+66.600689663" lastFinishedPulling="2026-06-03 13:48:30.175421067 +0000 UTC m=+67.946109782" observedRunningTime="2026-06-03 13:48:31.068919603 +0000 UTC m=+68.839608332" watchObservedRunningTime="2026-06-03 13:48:32.760132492 +0000 UTC m=+70.530821218" Jun 03 13:48:33.059210 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:33.059123 2573 patch_prober.go:28] interesting pod/image-registry-77b75f88c5-7smnt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 13:48:33.059210 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:33.059188 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" podUID="46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 13:48:35.968634 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:35.968602 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:48:38.069213 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.069168 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7ptv9" event={"ID":"72f705bc-93cc-4b36-bd92-ef3ad046561e","Type":"ContainerStarted","Data":"8c722541c021ac3a962ff3bf65cf7cd78ade36487afff91027543782cd09f11f"} Jun 03 13:48:38.071408 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.071358 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dn9ds" event={"ID":"3042a477-badc-4cb2-9b86-ae27a9649d57","Type":"ContainerStarted","Data":"6194738e319ebc72829b6d2ff0326a98862c401bd89337380ea7c71505ca1fcd"} Jun 03 13:48:38.072900 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.072864 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" event={"ID":"10fb9ad6-8d8a-4944-be96-428dd232c0a6","Type":"ContainerStarted","Data":"64acea42637df84d77f042e306f6a50f25766ff4b15a30711dcf77412ca151a2"} Jun 03 13:48:38.074330 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.074305 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l4mjx" event={"ID":"40b054a1-6cc3-4c1a-90b8-c780b8358de9","Type":"ContainerStarted","Data":"8b9857fe9a3914bcd65cff5321f6b311cc7549d2cb3c8bb0d8021f871709b7f2"} Jun 03 13:48:38.074450 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.074429 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:48:38.075619 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.075595 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" event={"ID":"01f6d122-ba9f-47f6-976c-c71587e64f2c","Type":"ContainerStarted","Data":"4b6a1a150c8ee1055be7df57d9a7a650e49f96a70fc3688856d6db14fc85b774"} Jun 03 13:48:38.075819 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.075801 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:38.077520 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.077491 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" Jun 03 13:48:38.087335 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.087280 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7ptv9" podStartSLOduration=33.09073524 podStartE2EDuration="41.087264947s" podCreationTimestamp="2026-06-03 13:47:57 +0000 UTC" firstStartedPulling="2026-06-03 13:48:29.421788736 +0000 UTC m=+67.192477453" lastFinishedPulling="2026-06-03 13:48:37.418318451 +0000 UTC m=+75.189007160" observedRunningTime="2026-06-03 13:48:38.086094652 +0000 UTC m=+75.856783389" watchObservedRunningTime="2026-06-03 13:48:38.087264947 +0000 UTC m=+75.857953678" Jun 03 13:48:38.107380 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.107330 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l4mjx" podStartSLOduration=67.516998885 podStartE2EDuration="1m16.107314817s" podCreationTimestamp="2026-06-03 13:47:22 +0000 UTC" firstStartedPulling="2026-06-03 13:48:28.828170374 +0000 UTC m=+66.598859077" lastFinishedPulling="2026-06-03 13:48:37.418486097 +0000 UTC m=+75.189175009" observedRunningTime="2026-06-03 13:48:38.105669453 +0000 UTC m=+75.876358179" watchObservedRunningTime="2026-06-03 13:48:38.107314817 +0000 UTC m=+75.878003542" Jun 03 13:48:38.132714 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.132640 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dn9ds" podStartSLOduration=1.798437234 podStartE2EDuration="10.132621618s" podCreationTimestamp="2026-06-03 13:48:28 +0000 UTC" firstStartedPulling="2026-06-03 13:48:29.085225174 +0000 UTC m=+66.855913876" lastFinishedPulling="2026-06-03 13:48:37.419409543 +0000 UTC m=+75.190098260" observedRunningTime="2026-06-03 13:48:38.132371746 +0000 UTC m=+75.903060472" watchObservedRunningTime="2026-06-03 13:48:38.132621618 +0000 UTC m=+75.903310344" Jun 03 13:48:38.159897 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.159849 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-645fb9d878-nqccs" podStartSLOduration=1.6201533399999999 podStartE2EDuration="10.15983372s" podCreationTimestamp="2026-06-03 13:48:28 +0000 UTC" firstStartedPulling="2026-06-03 13:48:28.903487339 +0000 UTC m=+66.674176042" lastFinishedPulling="2026-06-03 13:48:37.443167705 +0000 UTC m=+75.213856422" observedRunningTime="2026-06-03 13:48:38.158967773 +0000 UTC m=+75.929656507" watchObservedRunningTime="2026-06-03 13:48:38.15983372 +0000 UTC m=+75.930522442" Jun 03 13:48:38.180798 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:38.180739 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b6f9844db-x4snc" podStartSLOduration=1.682053113 podStartE2EDuration="10.180723678s" podCreationTimestamp="2026-06-03 13:48:28 +0000 UTC" firstStartedPulling="2026-06-03 13:48:28.932103023 +0000 UTC m=+66.702791739" lastFinishedPulling="2026-06-03 13:48:37.4307736 +0000 UTC m=+75.201462304" observedRunningTime="2026-06-03 13:48:38.179435946 +0000 UTC m=+75.950124671" watchObservedRunningTime="2026-06-03 13:48:38.180723678 +0000 UTC m=+75.951412404" Jun 03 13:48:39.499916 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.499855 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mgxs9"] Jun 03 13:48:39.505259 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.505228 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.510544 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.510519 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 13:48:39.511559 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.511536 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q77dj\"" Jun 03 13:48:39.511897 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.511860 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 13:48:39.512001 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.511870 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 13:48:39.514487 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.514463 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 13:48:39.514588 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.514525 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 13:48:39.526085 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.526067 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 13:48:39.661014 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.660970 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-sys\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.661182 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.661020 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-textfile\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.661182 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.661049 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-root\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.661182 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.661081 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.661182 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.661115 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-tls\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.661182 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.661170 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-wtmp\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.661394 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.661207 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-metrics-client-ca\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.661394 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.661245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.661394 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.661298 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppxr2\" (UniqueName: \"kubernetes.io/projected/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-kube-api-access-ppxr2\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762091 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762006 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-sys\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762091 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762050 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-textfile\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762091 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762074 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-root\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762129 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-root\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762126 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-sys\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762187 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762229 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-tls\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762251 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-wtmp\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762269 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-metrics-client-ca\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762291 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762311 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ppxr2\" (UniqueName: \"kubernetes.io/projected/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-kube-api-access-ppxr2\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762683 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762332 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-textfile\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762683 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762420 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-wtmp\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762683 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762674 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.762838 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.762770 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-metrics-client-ca\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.764778 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.764755 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-tls\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.764908 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.764865 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.770860 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.770838 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppxr2\" (UniqueName: \"kubernetes.io/projected/267ee72b-2613-4f92-bbb0-8af92e3ca4f2-kube-api-access-ppxr2\") pod \"node-exporter-mgxs9\" (UID: \"267ee72b-2613-4f92-bbb0-8af92e3ca4f2\") " pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.816846 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:39.816709 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mgxs9" Jun 03 13:48:39.828741 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:48:39.828645 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267ee72b_2613_4f92_bbb0_8af92e3ca4f2.slice/crio-36091e4ad3f7132bf83b4494615f5948c4eb708d8b395cfcfc7d29a07273ce1d WatchSource:0}: Error finding container 36091e4ad3f7132bf83b4494615f5948c4eb708d8b395cfcfc7d29a07273ce1d: Status 404 returned error can't find the container with id 36091e4ad3f7132bf83b4494615f5948c4eb708d8b395cfcfc7d29a07273ce1d Jun 03 13:48:40.082301 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:40.082220 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgxs9" event={"ID":"267ee72b-2613-4f92-bbb0-8af92e3ca4f2","Type":"ContainerStarted","Data":"36091e4ad3f7132bf83b4494615f5948c4eb708d8b395cfcfc7d29a07273ce1d"} Jun 03 13:48:41.086972 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:41.086862 2573 generic.go:358] "Generic (PLEG): container finished" podID="267ee72b-2613-4f92-bbb0-8af92e3ca4f2" containerID="8e93a9ce29d0e344f55606ee677e74e6924140742c229158e1b2ca700f906ff8" exitCode=0 Jun 03 13:48:41.086972 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:41.086947 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgxs9" event={"ID":"267ee72b-2613-4f92-bbb0-8af92e3ca4f2","Type":"ContainerDied","Data":"8e93a9ce29d0e344f55606ee677e74e6924140742c229158e1b2ca700f906ff8"} Jun 03 13:48:42.093803 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:42.093761 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgxs9" event={"ID":"267ee72b-2613-4f92-bbb0-8af92e3ca4f2","Type":"ContainerStarted","Data":"b04f5f3dbca05e12e17ca10669e39e65b9032c2f30540e1e227bca4b0dab0894"} Jun 03 13:48:42.094249 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:42.093809 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgxs9" event={"ID":"267ee72b-2613-4f92-bbb0-8af92e3ca4f2","Type":"ContainerStarted","Data":"1db6ed9b6ab0881facd28948f03d9dc7328aaeebea18d3abe626f86ceadd2bb8"} Jun 03 13:48:42.130561 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:42.130499 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mgxs9" podStartSLOduration=2.177869437 podStartE2EDuration="3.130481311s" podCreationTimestamp="2026-06-03 13:48:39 +0000 UTC" firstStartedPulling="2026-06-03 13:48:39.831123296 +0000 UTC m=+77.601812001" lastFinishedPulling="2026-06-03 13:48:40.783735158 +0000 UTC m=+78.554423875" observedRunningTime="2026-06-03 13:48:42.12849027 +0000 UTC m=+79.899178994" watchObservedRunningTime="2026-06-03 13:48:42.130481311 +0000 UTC m=+79.901170035" Jun 03 13:48:49.117123 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:49.117019 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-vtbnp" event={"ID":"4bef7fe1-c7ed-429f-bb84-80b0871b530c","Type":"ContainerStarted","Data":"3192b18a905b46bdc6b982dd4ae81b9fa1ec97e553c08dd9c30c1caac1b9a4f8"} Jun 03 13:48:49.117648 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:49.117355 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-vtbnp" Jun 03 13:48:49.131462 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:49.131426 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-vtbnp" Jun 03 13:48:49.140054 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:49.139994 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-vtbnp" podStartSLOduration=1.406597913 podStartE2EDuration="21.139979325s" podCreationTimestamp="2026-06-03 13:48:28 +0000 UTC" firstStartedPulling="2026-06-03 13:48:29.010032102 +0000 UTC m=+66.780720805" lastFinishedPulling="2026-06-03 13:48:48.743413513 +0000 UTC m=+86.514102217" observedRunningTime="2026-06-03 13:48:49.138452265 +0000 UTC m=+86.909140990" watchObservedRunningTime="2026-06-03 13:48:49.139979325 +0000 UTC m=+86.910668052" Jun 03 13:48:51.625335 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:48:51.625295 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77b75f88c5-7smnt"] Jun 03 13:49:09.082039 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:09.082001 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l4mjx" Jun 03 13:49:16.648791 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.648731 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" podUID="46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" containerName="registry" containerID="cri-o://dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c" gracePeriod=30 Jun 03 13:49:16.901217 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.901149 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:49:16.918327 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918303 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-installation-pull-secrets\") pod \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " Jun 03 13:49:16.918455 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918340 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-ca-trust-extracted\") pod \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " Jun 03 13:49:16.918455 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918362 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfqsz\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-kube-api-access-qfqsz\") pod \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " Jun 03 13:49:16.918455 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918397 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-image-registry-private-configuration\") pod \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " Jun 03 13:49:16.918455 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918432 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-bound-sa-token\") pod \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " Jun 03 13:49:16.918664 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918458 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-certificates\") pod \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " Jun 03 13:49:16.918664 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918499 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") pod \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " Jun 03 13:49:16.918817 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918791 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-trusted-ca\") pod \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\" (UID: \"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b\") " Jun 03 13:49:16.919027 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.918980 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 13:49:16.919152 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.919077 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-certificates\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:49:16.919408 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.919381 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 13:49:16.921016 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.920991 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:49:16.921133 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.921097 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-kube-api-access-qfqsz" (OuterVolumeSpecName: "kube-api-access-qfqsz") pod "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b"). InnerVolumeSpecName "kube-api-access-qfqsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:49:16.921196 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.921154 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:49:16.921323 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.921301 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 13:49:16.921390 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.921364 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 13:49:16.928556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:16.928530 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" (UID: "46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:49:17.020273 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.020239 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-registry-tls\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:49:17.020273 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.020266 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-trusted-ca\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:49:17.020273 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.020277 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-installation-pull-secrets\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:49:17.020499 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.020288 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-ca-trust-extracted\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:49:17.020499 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.020296 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfqsz\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-kube-api-access-qfqsz\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:49:17.020499 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.020307 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-image-registry-private-configuration\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:49:17.020499 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.020316 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b-bound-sa-token\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:49:17.195047 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.194958 2573 generic.go:358] "Generic (PLEG): container finished" podID="46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" containerID="dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c" exitCode=0 Jun 03 13:49:17.195047 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.195022 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" event={"ID":"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b","Type":"ContainerDied","Data":"dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c"} Jun 03 13:49:17.195047 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.195048 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" event={"ID":"46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b","Type":"ContainerDied","Data":"32a55e155e0d00738a74ee8991920aafcb63b2a803449952e556751af4d2a705"} Jun 03 13:49:17.195305 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.195055 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77b75f88c5-7smnt" Jun 03 13:49:17.195305 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.195062 2573 scope.go:117] "RemoveContainer" containerID="dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c" Jun 03 13:49:17.203270 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.203250 2573 scope.go:117] "RemoveContainer" containerID="dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c" Jun 03 13:49:17.203584 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:49:17.203548 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c\": container with ID starting with dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c not found: ID does not exist" containerID="dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c" Jun 03 13:49:17.203677 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.203591 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c"} err="failed to get container status \"dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c\": rpc error: code = NotFound desc = could not find container \"dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c\": container with ID starting with dd5de0fb8b2fa188ea2791a05cb5851fba83c5b490746978a17eb94025e1189c not found: ID does not exist" Jun 03 13:49:17.251675 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.251644 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77b75f88c5-7smnt"] Jun 03 13:49:17.274468 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.274418 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77b75f88c5-7smnt"] Jun 03 13:49:17.334178 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:17.334151 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7ptv9_72f705bc-93cc-4b36-bd92-ef3ad046561e/serve-healthcheck-canary/0.log" Jun 03 13:49:18.737897 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:49:18.737862 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" path="/var/lib/kubelet/pods/46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b/volumes" Jun 03 13:52:22.646779 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:22.646749 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 13:52:22.647270 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:22.646753 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 13:52:22.655712 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:22.655687 2573 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 13:52:27.846037 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.846001 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg"] Jun 03 13:52:27.847414 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.846336 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" containerName="registry" Jun 03 13:52:27.847414 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.846351 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" containerName="registry" Jun 03 13:52:27.847414 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.846424 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="46f5f71e-0cdf-44ba-a5b8-3b1fb74e494b" containerName="registry" Jun 03 13:52:27.848070 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.848054 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:27.850350 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.850326 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 13:52:27.850504 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.850476 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 13:52:27.851316 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.851300 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vf5vf\"" Jun 03 13:52:27.858104 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.858076 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg"] Jun 03 13:52:27.897005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.896979 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:27.897094 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.897025 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:27.897094 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.897079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlhvd\" (UniqueName: \"kubernetes.io/projected/da24c877-f033-473b-96d4-84e994fce21e-kube-api-access-qlhvd\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:27.997382 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.997347 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:27.997509 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.997394 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qlhvd\" (UniqueName: \"kubernetes.io/projected/da24c877-f033-473b-96d4-84e994fce21e-kube-api-access-qlhvd\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:27.997509 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.997428 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:27.997738 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.997716 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:27.997794 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:27.997778 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:28.005773 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:28.005745 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlhvd\" (UniqueName: \"kubernetes.io/projected/da24c877-f033-473b-96d4-84e994fce21e-kube-api-access-qlhvd\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:28.157824 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:28.157800 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:28.278816 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:28.278775 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg"] Jun 03 13:52:28.283682 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:52:28.283646 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda24c877_f033_473b_96d4_84e994fce21e.slice/crio-e52ac7bbd8165e96a25713a1f3c20fad68c53cf13cc751e8c2fb2c6428e20a10 WatchSource:0}: Error finding container e52ac7bbd8165e96a25713a1f3c20fad68c53cf13cc751e8c2fb2c6428e20a10: Status 404 returned error can't find the container with id e52ac7bbd8165e96a25713a1f3c20fad68c53cf13cc751e8c2fb2c6428e20a10 Jun 03 13:52:28.285153 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:28.285137 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 13:52:28.671968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:28.671932 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" event={"ID":"da24c877-f033-473b-96d4-84e994fce21e","Type":"ContainerStarted","Data":"e52ac7bbd8165e96a25713a1f3c20fad68c53cf13cc751e8c2fb2c6428e20a10"} Jun 03 13:52:35.692422 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:35.692397 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" event={"ID":"da24c877-f033-473b-96d4-84e994fce21e","Type":"ContainerStarted","Data":"3245bcf63b15d63920ee81a7f02b9e9b77d67db8b609aaf8e9f745da3578fdfb"} Jun 03 13:52:36.696770 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:36.696731 2573 generic.go:358] "Generic (PLEG): container finished" podID="da24c877-f033-473b-96d4-84e994fce21e" containerID="3245bcf63b15d63920ee81a7f02b9e9b77d67db8b609aaf8e9f745da3578fdfb" exitCode=0 Jun 03 13:52:36.697181 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:36.696787 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" event={"ID":"da24c877-f033-473b-96d4-84e994fce21e","Type":"ContainerDied","Data":"3245bcf63b15d63920ee81a7f02b9e9b77d67db8b609aaf8e9f745da3578fdfb"} Jun 03 13:52:38.704742 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:38.704712 2573 generic.go:358] "Generic (PLEG): container finished" podID="da24c877-f033-473b-96d4-84e994fce21e" containerID="2eadc5ca363694fb2c50f164db0c0b00ae6483f70fab1aded2d4a56930dc3246" exitCode=0 Jun 03 13:52:38.705109 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:38.704781 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" event={"ID":"da24c877-f033-473b-96d4-84e994fce21e","Type":"ContainerDied","Data":"2eadc5ca363694fb2c50f164db0c0b00ae6483f70fab1aded2d4a56930dc3246"} Jun 03 13:52:45.724071 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:45.724002 2573 generic.go:358] "Generic (PLEG): container finished" podID="da24c877-f033-473b-96d4-84e994fce21e" containerID="2dc83f59ffec5dc8cbf5a2a78127dcec1ad3df1034abf7baa3945a5b88668935" exitCode=0 Jun 03 13:52:45.724373 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:45.724100 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" event={"ID":"da24c877-f033-473b-96d4-84e994fce21e","Type":"ContainerDied","Data":"2dc83f59ffec5dc8cbf5a2a78127dcec1ad3df1034abf7baa3945a5b88668935"} Jun 03 13:52:46.844683 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:46.844659 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:46.940177 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:46.940149 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlhvd\" (UniqueName: \"kubernetes.io/projected/da24c877-f033-473b-96d4-84e994fce21e-kube-api-access-qlhvd\") pod \"da24c877-f033-473b-96d4-84e994fce21e\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " Jun 03 13:52:46.940325 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:46.940229 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-bundle\") pod \"da24c877-f033-473b-96d4-84e994fce21e\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " Jun 03 13:52:46.940325 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:46.940273 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-util\") pod \"da24c877-f033-473b-96d4-84e994fce21e\" (UID: \"da24c877-f033-473b-96d4-84e994fce21e\") " Jun 03 13:52:46.940833 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:46.940802 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-bundle" (OuterVolumeSpecName: "bundle") pod "da24c877-f033-473b-96d4-84e994fce21e" (UID: "da24c877-f033-473b-96d4-84e994fce21e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:52:46.942252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:46.942232 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da24c877-f033-473b-96d4-84e994fce21e-kube-api-access-qlhvd" (OuterVolumeSpecName: "kube-api-access-qlhvd") pod "da24c877-f033-473b-96d4-84e994fce21e" (UID: "da24c877-f033-473b-96d4-84e994fce21e"). InnerVolumeSpecName "kube-api-access-qlhvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:52:46.944655 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:46.944628 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-util" (OuterVolumeSpecName: "util") pod "da24c877-f033-473b-96d4-84e994fce21e" (UID: "da24c877-f033-473b-96d4-84e994fce21e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:52:47.041619 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:47.041547 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:52:47.041619 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:47.041573 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da24c877-f033-473b-96d4-84e994fce21e-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:52:47.041619 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:47.041581 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlhvd\" (UniqueName: \"kubernetes.io/projected/da24c877-f033-473b-96d4-84e994fce21e-kube-api-access-qlhvd\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:52:47.730899 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:47.730865 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" Jun 03 13:52:47.731065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:47.730864 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dcsnlg" event={"ID":"da24c877-f033-473b-96d4-84e994fce21e","Type":"ContainerDied","Data":"e52ac7bbd8165e96a25713a1f3c20fad68c53cf13cc751e8c2fb2c6428e20a10"} Jun 03 13:52:47.731065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:47.730981 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e52ac7bbd8165e96a25713a1f3c20fad68c53cf13cc751e8c2fb2c6428e20a10" Jun 03 13:52:50.905593 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.905561 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72"] Jun 03 13:52:50.905974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.905782 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da24c877-f033-473b-96d4-84e994fce21e" containerName="pull" Jun 03 13:52:50.905974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.905792 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="da24c877-f033-473b-96d4-84e994fce21e" containerName="pull" Jun 03 13:52:50.905974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.905801 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da24c877-f033-473b-96d4-84e994fce21e" containerName="extract" Jun 03 13:52:50.905974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.905807 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="da24c877-f033-473b-96d4-84e994fce21e" containerName="extract" Jun 03 13:52:50.905974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.905820 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da24c877-f033-473b-96d4-84e994fce21e" containerName="util" Jun 03 13:52:50.905974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.905825 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="da24c877-f033-473b-96d4-84e994fce21e" containerName="util" Jun 03 13:52:50.905974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.905869 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="da24c877-f033-473b-96d4-84e994fce21e" containerName="extract" Jun 03 13:52:50.958486 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.958455 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72"] Jun 03 13:52:50.958632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.958573 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" Jun 03 13:52:50.961128 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.961103 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 03 13:52:50.961431 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.961410 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 03 13:52:50.961711 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:50.961694 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-wl9f2\"" Jun 03 13:52:51.072831 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.072797 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34a1bfb2-9e89-4c98-bd48-0095cc00b3df-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-zzs72\" (UID: \"34a1bfb2-9e89-4c98-bd48-0095cc00b3df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" Jun 03 13:52:51.072987 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.072862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svlhp\" (UniqueName: \"kubernetes.io/projected/34a1bfb2-9e89-4c98-bd48-0095cc00b3df-kube-api-access-svlhp\") pod \"cert-manager-operator-controller-manager-54b9655956-zzs72\" (UID: \"34a1bfb2-9e89-4c98-bd48-0095cc00b3df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" Jun 03 13:52:51.173813 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.173742 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-svlhp\" (UniqueName: \"kubernetes.io/projected/34a1bfb2-9e89-4c98-bd48-0095cc00b3df-kube-api-access-svlhp\") pod \"cert-manager-operator-controller-manager-54b9655956-zzs72\" (UID: \"34a1bfb2-9e89-4c98-bd48-0095cc00b3df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" Jun 03 13:52:51.173936 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.173832 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34a1bfb2-9e89-4c98-bd48-0095cc00b3df-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-zzs72\" (UID: \"34a1bfb2-9e89-4c98-bd48-0095cc00b3df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" Jun 03 13:52:51.174251 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.174233 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34a1bfb2-9e89-4c98-bd48-0095cc00b3df-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-zzs72\" (UID: \"34a1bfb2-9e89-4c98-bd48-0095cc00b3df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" Jun 03 13:52:51.182939 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.182914 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svlhp\" (UniqueName: \"kubernetes.io/projected/34a1bfb2-9e89-4c98-bd48-0095cc00b3df-kube-api-access-svlhp\") pod \"cert-manager-operator-controller-manager-54b9655956-zzs72\" (UID: \"34a1bfb2-9e89-4c98-bd48-0095cc00b3df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" Jun 03 13:52:51.267864 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.267832 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" Jun 03 13:52:51.397496 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.397468 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72"] Jun 03 13:52:51.402769 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:52:51.402734 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a1bfb2_9e89_4c98_bd48_0095cc00b3df.slice/crio-12c7cafce240931071f112f130b899f35cdda52a24d4b4d8ac308334d503e73d WatchSource:0}: Error finding container 12c7cafce240931071f112f130b899f35cdda52a24d4b4d8ac308334d503e73d: Status 404 returned error can't find the container with id 12c7cafce240931071f112f130b899f35cdda52a24d4b4d8ac308334d503e73d Jun 03 13:52:51.742556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:51.742523 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" event={"ID":"34a1bfb2-9e89-4c98-bd48-0095cc00b3df","Type":"ContainerStarted","Data":"12c7cafce240931071f112f130b899f35cdda52a24d4b4d8ac308334d503e73d"} Jun 03 13:52:54.750636 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:54.750595 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" event={"ID":"34a1bfb2-9e89-4c98-bd48-0095cc00b3df","Type":"ContainerStarted","Data":"b62b69ed6a3a41c115999352f2e43db5484f02b576e2ce3c2e670ae1d88a0bb9"} Jun 03 13:52:54.773271 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:54.773220 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-zzs72" podStartSLOduration=2.264906082 podStartE2EDuration="4.773208557s" podCreationTimestamp="2026-06-03 13:52:50 +0000 UTC" firstStartedPulling="2026-06-03 13:52:51.40501014 +0000 UTC m=+329.175698854" lastFinishedPulling="2026-06-03 13:52:53.913312619 +0000 UTC m=+331.684001329" observedRunningTime="2026-06-03 13:52:54.773046752 +0000 UTC m=+332.543735476" watchObservedRunningTime="2026-06-03 13:52:54.773208557 +0000 UTC m=+332.543897282" Jun 03 13:52:55.498425 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.498391 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc"] Jun 03 13:52:55.500590 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.500573 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.503083 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.502987 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 13:52:55.503083 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.502993 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 13:52:55.503250 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.503059 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vf5vf\"" Jun 03 13:52:55.512152 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.512134 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc"] Jun 03 13:52:55.604339 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.604304 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.604494 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.604352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.604494 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.604373 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxqwq\" (UniqueName: \"kubernetes.io/projected/4725b953-dd3e-4fd0-9957-a1ce73083aec-kube-api-access-vxqwq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.705756 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.705721 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.705909 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.705760 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vxqwq\" (UniqueName: \"kubernetes.io/projected/4725b953-dd3e-4fd0-9957-a1ce73083aec-kube-api-access-vxqwq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.705909 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.705800 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.706121 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.706103 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.706170 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.706130 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.713760 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.713738 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxqwq\" (UniqueName: \"kubernetes.io/projected/4725b953-dd3e-4fd0-9957-a1ce73083aec-kube-api-access-vxqwq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.809039 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.808961 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:52:55.926170 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:55.926072 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc"] Jun 03 13:52:55.928510 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:52:55.928477 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4725b953_dd3e_4fd0_9957_a1ce73083aec.slice/crio-cd18b62941d918c3ab8a6438e8e4b5a1615b1b8cf79a4923d4c2c9d5d4e01c75 WatchSource:0}: Error finding container cd18b62941d918c3ab8a6438e8e4b5a1615b1b8cf79a4923d4c2c9d5d4e01c75: Status 404 returned error can't find the container with id cd18b62941d918c3ab8a6438e8e4b5a1615b1b8cf79a4923d4c2c9d5d4e01c75 Jun 03 13:52:56.756694 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:56.756660 2573 generic.go:358] "Generic (PLEG): container finished" podID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerID="35cd3f23ccc6288790df752a3f0e410cc54f999bf8b111f6b88b0184d767f5e7" exitCode=0 Jun 03 13:52:56.756821 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:56.756744 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" event={"ID":"4725b953-dd3e-4fd0-9957-a1ce73083aec","Type":"ContainerDied","Data":"35cd3f23ccc6288790df752a3f0e410cc54f999bf8b111f6b88b0184d767f5e7"} Jun 03 13:52:56.756821 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:56.756773 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" event={"ID":"4725b953-dd3e-4fd0-9957-a1ce73083aec","Type":"ContainerStarted","Data":"cd18b62941d918c3ab8a6438e8e4b5a1615b1b8cf79a4923d4c2c9d5d4e01c75"} Jun 03 13:52:57.687065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.687030 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-zn67t"] Jun 03 13:52:57.688858 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.688842 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:52:57.691342 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.691319 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 03 13:52:57.692143 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.692128 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-gltf4\"" Jun 03 13:52:57.692143 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.692136 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 03 13:52:57.698763 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.698717 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-zn67t"] Jun 03 13:52:57.820833 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.820804 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f7cd32e-d3b8-424b-b204-9fb1b6aca66f-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-zn67t\" (UID: \"1f7cd32e-d3b8-424b-b204-9fb1b6aca66f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:52:57.821008 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.820861 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk7gv\" (UniqueName: \"kubernetes.io/projected/1f7cd32e-d3b8-424b-b204-9fb1b6aca66f-kube-api-access-qk7gv\") pod \"cert-manager-webhook-587ccfb98-zn67t\" (UID: \"1f7cd32e-d3b8-424b-b204-9fb1b6aca66f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:52:57.921340 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.921310 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f7cd32e-d3b8-424b-b204-9fb1b6aca66f-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-zn67t\" (UID: \"1f7cd32e-d3b8-424b-b204-9fb1b6aca66f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:52:57.921488 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.921365 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qk7gv\" (UniqueName: \"kubernetes.io/projected/1f7cd32e-d3b8-424b-b204-9fb1b6aca66f-kube-api-access-qk7gv\") pod \"cert-manager-webhook-587ccfb98-zn67t\" (UID: \"1f7cd32e-d3b8-424b-b204-9fb1b6aca66f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:52:57.929705 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.929659 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f7cd32e-d3b8-424b-b204-9fb1b6aca66f-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-zn67t\" (UID: \"1f7cd32e-d3b8-424b-b204-9fb1b6aca66f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:52:57.930140 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.930120 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk7gv\" (UniqueName: \"kubernetes.io/projected/1f7cd32e-d3b8-424b-b204-9fb1b6aca66f-kube-api-access-qk7gv\") pod \"cert-manager-webhook-587ccfb98-zn67t\" (UID: \"1f7cd32e-d3b8-424b-b204-9fb1b6aca66f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:52:57.998091 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:57.997996 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:52:58.120124 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:58.120094 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-zn67t"] Jun 03 13:52:58.122807 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:52:58.122778 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f7cd32e_d3b8_424b_b204_9fb1b6aca66f.slice/crio-8c09f066b7730981ac55889cdcf855837a33a6692bb29afe465429b8935ad630 WatchSource:0}: Error finding container 8c09f066b7730981ac55889cdcf855837a33a6692bb29afe465429b8935ad630: Status 404 returned error can't find the container with id 8c09f066b7730981ac55889cdcf855837a33a6692bb29afe465429b8935ad630 Jun 03 13:52:58.762992 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:58.762956 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" event={"ID":"1f7cd32e-d3b8-424b-b204-9fb1b6aca66f","Type":"ContainerStarted","Data":"8c09f066b7730981ac55889cdcf855837a33a6692bb29afe465429b8935ad630"} Jun 03 13:52:59.989125 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:59.989087 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-zm6bv"] Jun 03 13:52:59.991107 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:59.991091 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" Jun 03 13:52:59.993347 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:52:59.993323 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8wspw\"" Jun 03 13:53:00.002838 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.002811 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-zm6bv"] Jun 03 13:53:00.135866 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.135833 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/933c97ad-00ca-483e-9d64-b66eba10ad90-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-zm6bv\" (UID: \"933c97ad-00ca-483e-9d64-b66eba10ad90\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" Jun 03 13:53:00.136077 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.135928 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmtq4\" (UniqueName: \"kubernetes.io/projected/933c97ad-00ca-483e-9d64-b66eba10ad90-kube-api-access-jmtq4\") pod \"cert-manager-cainjector-68b757865b-zm6bv\" (UID: \"933c97ad-00ca-483e-9d64-b66eba10ad90\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" Jun 03 13:53:00.237197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.237158 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jmtq4\" (UniqueName: \"kubernetes.io/projected/933c97ad-00ca-483e-9d64-b66eba10ad90-kube-api-access-jmtq4\") pod \"cert-manager-cainjector-68b757865b-zm6bv\" (UID: \"933c97ad-00ca-483e-9d64-b66eba10ad90\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" Jun 03 13:53:00.237370 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.237232 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/933c97ad-00ca-483e-9d64-b66eba10ad90-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-zm6bv\" (UID: \"933c97ad-00ca-483e-9d64-b66eba10ad90\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" Jun 03 13:53:00.245819 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.245762 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/933c97ad-00ca-483e-9d64-b66eba10ad90-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-zm6bv\" (UID: \"933c97ad-00ca-483e-9d64-b66eba10ad90\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" Jun 03 13:53:00.246336 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.246313 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmtq4\" (UniqueName: \"kubernetes.io/projected/933c97ad-00ca-483e-9d64-b66eba10ad90-kube-api-access-jmtq4\") pod \"cert-manager-cainjector-68b757865b-zm6bv\" (UID: \"933c97ad-00ca-483e-9d64-b66eba10ad90\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" Jun 03 13:53:00.301108 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.301066 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" Jun 03 13:53:00.419544 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.419508 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-zm6bv"] Jun 03 13:53:00.423418 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:53:00.423392 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod933c97ad_00ca_483e_9d64_b66eba10ad90.slice/crio-929fa8feed5f2e38eac39c33fdb63e8f83d1dc6605beed1c58cc44e4a585dea1 WatchSource:0}: Error finding container 929fa8feed5f2e38eac39c33fdb63e8f83d1dc6605beed1c58cc44e4a585dea1: Status 404 returned error can't find the container with id 929fa8feed5f2e38eac39c33fdb63e8f83d1dc6605beed1c58cc44e4a585dea1 Jun 03 13:53:00.769616 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:00.769585 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" event={"ID":"933c97ad-00ca-483e-9d64-b66eba10ad90","Type":"ContainerStarted","Data":"929fa8feed5f2e38eac39c33fdb63e8f83d1dc6605beed1c58cc44e4a585dea1"} Jun 03 13:53:04.781335 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:04.781255 2573 generic.go:358] "Generic (PLEG): container finished" podID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerID="b4f09429cff8b6b9e8144304112ac4ea97eb9873f436c9c7d082c5c8b6c71b59" exitCode=0 Jun 03 13:53:04.781668 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:04.781337 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" event={"ID":"4725b953-dd3e-4fd0-9957-a1ce73083aec","Type":"ContainerDied","Data":"b4f09429cff8b6b9e8144304112ac4ea97eb9873f436c9c7d082c5c8b6c71b59"} Jun 03 13:53:05.786154 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:05.786126 2573 generic.go:358] "Generic (PLEG): container finished" podID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerID="b736eda31524b33779b574d3b3053f42a279c5b372d7974b8e392995cf7dcee5" exitCode=0 Jun 03 13:53:05.786529 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:05.786203 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" event={"ID":"4725b953-dd3e-4fd0-9957-a1ce73083aec","Type":"ContainerDied","Data":"b736eda31524b33779b574d3b3053f42a279c5b372d7974b8e392995cf7dcee5"} Jun 03 13:53:07.759391 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.759370 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:53:07.793681 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.793657 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" Jun 03 13:53:07.793794 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.793652 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4msxc" event={"ID":"4725b953-dd3e-4fd0-9957-a1ce73083aec","Type":"ContainerDied","Data":"cd18b62941d918c3ab8a6438e8e4b5a1615b1b8cf79a4923d4c2c9d5d4e01c75"} Jun 03 13:53:07.793794 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.793764 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd18b62941d918c3ab8a6438e8e4b5a1615b1b8cf79a4923d4c2c9d5d4e01c75" Jun 03 13:53:07.893375 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.893353 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-util\") pod \"4725b953-dd3e-4fd0-9957-a1ce73083aec\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " Jun 03 13:53:07.893460 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.893385 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-bundle\") pod \"4725b953-dd3e-4fd0-9957-a1ce73083aec\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " Jun 03 13:53:07.893460 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.893406 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxqwq\" (UniqueName: \"kubernetes.io/projected/4725b953-dd3e-4fd0-9957-a1ce73083aec-kube-api-access-vxqwq\") pod \"4725b953-dd3e-4fd0-9957-a1ce73083aec\" (UID: \"4725b953-dd3e-4fd0-9957-a1ce73083aec\") " Jun 03 13:53:07.893739 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.893716 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-bundle" (OuterVolumeSpecName: "bundle") pod "4725b953-dd3e-4fd0-9957-a1ce73083aec" (UID: "4725b953-dd3e-4fd0-9957-a1ce73083aec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:53:07.895531 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.895504 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4725b953-dd3e-4fd0-9957-a1ce73083aec-kube-api-access-vxqwq" (OuterVolumeSpecName: "kube-api-access-vxqwq") pod "4725b953-dd3e-4fd0-9957-a1ce73083aec" (UID: "4725b953-dd3e-4fd0-9957-a1ce73083aec"). InnerVolumeSpecName "kube-api-access-vxqwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:53:07.900084 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.900054 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-util" (OuterVolumeSpecName: "util") pod "4725b953-dd3e-4fd0-9957-a1ce73083aec" (UID: "4725b953-dd3e-4fd0-9957-a1ce73083aec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:53:07.994148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.994115 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:53:07.994148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.994144 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4725b953-dd3e-4fd0-9957-a1ce73083aec-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:53:07.994148 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:07.994153 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxqwq\" (UniqueName: \"kubernetes.io/projected/4725b953-dd3e-4fd0-9957-a1ce73083aec-kube-api-access-vxqwq\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:53:08.796987 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:08.796942 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" event={"ID":"1f7cd32e-d3b8-424b-b204-9fb1b6aca66f","Type":"ContainerStarted","Data":"0b4083a7cb85a73955ac700724106d40b17ae28de5a0a14c781976b92cc90659"} Jun 03 13:53:08.797424 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:08.797060 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:53:08.798321 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:08.798297 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" event={"ID":"933c97ad-00ca-483e-9d64-b66eba10ad90","Type":"ContainerStarted","Data":"ca8907fa26c270b6b55fb5fcaf2aea6878dd869ac2c3aee176bdfcf73882f5cf"} Jun 03 13:53:08.816207 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:08.816171 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" podStartSLOduration=2.121449616 podStartE2EDuration="11.816157485s" podCreationTimestamp="2026-06-03 13:52:57 +0000 UTC" firstStartedPulling="2026-06-03 13:52:58.124566457 +0000 UTC m=+335.895255159" lastFinishedPulling="2026-06-03 13:53:07.819274316 +0000 UTC m=+345.589963028" observedRunningTime="2026-06-03 13:53:08.814676114 +0000 UTC m=+346.585364839" watchObservedRunningTime="2026-06-03 13:53:08.816157485 +0000 UTC m=+346.586846209" Jun 03 13:53:08.831139 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:08.831101 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-zm6bv" podStartSLOduration=2.431281907 podStartE2EDuration="9.831088561s" podCreationTimestamp="2026-06-03 13:52:59 +0000 UTC" firstStartedPulling="2026-06-03 13:53:00.425205131 +0000 UTC m=+338.195893837" lastFinishedPulling="2026-06-03 13:53:07.825011775 +0000 UTC m=+345.595700491" observedRunningTime="2026-06-03 13:53:08.830007178 +0000 UTC m=+346.600695903" watchObservedRunningTime="2026-06-03 13:53:08.831088561 +0000 UTC m=+346.601777282" Jun 03 13:53:14.803718 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:14.803682 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-zn67t" Jun 03 13:53:15.355381 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.355346 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-7wtgb"] Jun 03 13:53:15.355603 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.355590 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerName="util" Jun 03 13:53:15.355651 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.355604 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerName="util" Jun 03 13:53:15.355651 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.355612 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerName="pull" Jun 03 13:53:15.355651 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.355618 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerName="pull" Jun 03 13:53:15.355651 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.355623 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerName="extract" Jun 03 13:53:15.355651 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.355628 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerName="extract" Jun 03 13:53:15.355790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.355671 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="4725b953-dd3e-4fd0-9957-a1ce73083aec" containerName="extract" Jun 03 13:53:15.373419 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.373392 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-7wtgb"] Jun 03 13:53:15.373542 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.373516 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-7wtgb" Jun 03 13:53:15.375788 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.375767 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lcdwt\"" Jun 03 13:53:15.447289 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.447264 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xwxg\" (UniqueName: \"kubernetes.io/projected/16fcd772-386e-46bf-8d02-7f5a58630d66-kube-api-access-2xwxg\") pod \"cert-manager-79c8d999ff-7wtgb\" (UID: \"16fcd772-386e-46bf-8d02-7f5a58630d66\") " pod="cert-manager/cert-manager-79c8d999ff-7wtgb" Jun 03 13:53:15.447410 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.447294 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16fcd772-386e-46bf-8d02-7f5a58630d66-bound-sa-token\") pod \"cert-manager-79c8d999ff-7wtgb\" (UID: \"16fcd772-386e-46bf-8d02-7f5a58630d66\") " pod="cert-manager/cert-manager-79c8d999ff-7wtgb" Jun 03 13:53:15.548101 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.548074 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2xwxg\" (UniqueName: \"kubernetes.io/projected/16fcd772-386e-46bf-8d02-7f5a58630d66-kube-api-access-2xwxg\") pod \"cert-manager-79c8d999ff-7wtgb\" (UID: \"16fcd772-386e-46bf-8d02-7f5a58630d66\") " pod="cert-manager/cert-manager-79c8d999ff-7wtgb" Jun 03 13:53:15.548213 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.548110 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16fcd772-386e-46bf-8d02-7f5a58630d66-bound-sa-token\") pod \"cert-manager-79c8d999ff-7wtgb\" (UID: \"16fcd772-386e-46bf-8d02-7f5a58630d66\") " pod="cert-manager/cert-manager-79c8d999ff-7wtgb" Jun 03 13:53:15.556929 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.556908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16fcd772-386e-46bf-8d02-7f5a58630d66-bound-sa-token\") pod \"cert-manager-79c8d999ff-7wtgb\" (UID: \"16fcd772-386e-46bf-8d02-7f5a58630d66\") " pod="cert-manager/cert-manager-79c8d999ff-7wtgb" Jun 03 13:53:15.557513 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.557494 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xwxg\" (UniqueName: \"kubernetes.io/projected/16fcd772-386e-46bf-8d02-7f5a58630d66-kube-api-access-2xwxg\") pod \"cert-manager-79c8d999ff-7wtgb\" (UID: \"16fcd772-386e-46bf-8d02-7f5a58630d66\") " pod="cert-manager/cert-manager-79c8d999ff-7wtgb" Jun 03 13:53:15.682633 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.682609 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-7wtgb" Jun 03 13:53:15.804040 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.803939 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-7wtgb"] Jun 03 13:53:15.806742 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:53:15.806715 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16fcd772_386e_46bf_8d02_7f5a58630d66.slice/crio-51aac5a996679f46c23c18918aa78a646903bca8b8dd2a6561c9fb606d239f3a WatchSource:0}: Error finding container 51aac5a996679f46c23c18918aa78a646903bca8b8dd2a6561c9fb606d239f3a: Status 404 returned error can't find the container with id 51aac5a996679f46c23c18918aa78a646903bca8b8dd2a6561c9fb606d239f3a Jun 03 13:53:15.818759 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:15.818732 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-7wtgb" event={"ID":"16fcd772-386e-46bf-8d02-7f5a58630d66","Type":"ContainerStarted","Data":"51aac5a996679f46c23c18918aa78a646903bca8b8dd2a6561c9fb606d239f3a"} Jun 03 13:53:16.823388 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:16.823355 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-7wtgb" event={"ID":"16fcd772-386e-46bf-8d02-7f5a58630d66","Type":"ContainerStarted","Data":"8c31872301ae57a02e8504e8a6979ebf62d9f72709e035dbc8e42accfc1d8dc7"} Jun 03 13:53:16.839263 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:16.839218 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-7wtgb" podStartSLOduration=1.839203601 podStartE2EDuration="1.839203601s" podCreationTimestamp="2026-06-03 13:53:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:53:16.837664265 +0000 UTC m=+354.608352990" watchObservedRunningTime="2026-06-03 13:53:16.839203601 +0000 UTC m=+354.609892325" Jun 03 13:53:20.449241 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.449198 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj"] Jun 03 13:53:20.451365 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.451349 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.453996 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.453977 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 13:53:20.454791 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.454770 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vf5vf\"" Jun 03 13:53:20.454870 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.454828 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 13:53:20.465452 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.465429 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj"] Jun 03 13:53:20.479863 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.479841 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb6pm\" (UniqueName: \"kubernetes.io/projected/eae75cde-0bd4-43af-9228-a043514f9f29-kube-api-access-kb6pm\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.479985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.479897 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.479985 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.479961 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.580766 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.580727 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.580974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.580804 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kb6pm\" (UniqueName: \"kubernetes.io/projected/eae75cde-0bd4-43af-9228-a043514f9f29-kube-api-access-kb6pm\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.580974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.580837 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.581179 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.581156 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.581244 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.581183 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.589457 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.589427 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb6pm\" (UniqueName: \"kubernetes.io/projected/eae75cde-0bd4-43af-9228-a043514f9f29-kube-api-access-kb6pm\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.760993 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.760917 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:20.899966 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:20.899934 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj"] Jun 03 13:53:20.902587 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:53:20.902559 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae75cde_0bd4_43af_9228_a043514f9f29.slice/crio-9d599996954454631c1c9a924ba1bf209e721dd76b5bd435527ee3f29e309bb3 WatchSource:0}: Error finding container 9d599996954454631c1c9a924ba1bf209e721dd76b5bd435527ee3f29e309bb3: Status 404 returned error can't find the container with id 9d599996954454631c1c9a924ba1bf209e721dd76b5bd435527ee3f29e309bb3 Jun 03 13:53:21.841407 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:21.841376 2573 generic.go:358] "Generic (PLEG): container finished" podID="eae75cde-0bd4-43af-9228-a043514f9f29" containerID="688bb7976433758b8c61cfcbb8c1da6f118c18eb89aa41cc0ad466125cc6150d" exitCode=0 Jun 03 13:53:21.841781 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:21.841420 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" event={"ID":"eae75cde-0bd4-43af-9228-a043514f9f29","Type":"ContainerDied","Data":"688bb7976433758b8c61cfcbb8c1da6f118c18eb89aa41cc0ad466125cc6150d"} Jun 03 13:53:21.841781 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:21.841460 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" event={"ID":"eae75cde-0bd4-43af-9228-a043514f9f29","Type":"ContainerStarted","Data":"9d599996954454631c1c9a924ba1bf209e721dd76b5bd435527ee3f29e309bb3"} Jun 03 13:53:22.845734 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:22.845656 2573 generic.go:358] "Generic (PLEG): container finished" podID="eae75cde-0bd4-43af-9228-a043514f9f29" containerID="c6626a6d9b5d3d5ef3d851e6f23dacba5555c2718e93451c281bfe34cac24e6d" exitCode=0 Jun 03 13:53:22.846122 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:22.845746 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" event={"ID":"eae75cde-0bd4-43af-9228-a043514f9f29","Type":"ContainerDied","Data":"c6626a6d9b5d3d5ef3d851e6f23dacba5555c2718e93451c281bfe34cac24e6d"} Jun 03 13:53:23.854686 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:23.854654 2573 generic.go:358] "Generic (PLEG): container finished" podID="eae75cde-0bd4-43af-9228-a043514f9f29" containerID="2492babf6cd482d0758332c8f76f592e06caf0f91e42c4d95bf7522d919dc222" exitCode=0 Jun 03 13:53:23.855095 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:23.854746 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" event={"ID":"eae75cde-0bd4-43af-9228-a043514f9f29","Type":"ContainerDied","Data":"2492babf6cd482d0758332c8f76f592e06caf0f91e42c4d95bf7522d919dc222"} Jun 03 13:53:24.978433 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:24.978409 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:25.010949 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.010919 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-util\") pod \"eae75cde-0bd4-43af-9228-a043514f9f29\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " Jun 03 13:53:25.011080 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.010956 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-bundle\") pod \"eae75cde-0bd4-43af-9228-a043514f9f29\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " Jun 03 13:53:25.011080 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.010975 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb6pm\" (UniqueName: \"kubernetes.io/projected/eae75cde-0bd4-43af-9228-a043514f9f29-kube-api-access-kb6pm\") pod \"eae75cde-0bd4-43af-9228-a043514f9f29\" (UID: \"eae75cde-0bd4-43af-9228-a043514f9f29\") " Jun 03 13:53:25.011788 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.011756 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-bundle" (OuterVolumeSpecName: "bundle") pod "eae75cde-0bd4-43af-9228-a043514f9f29" (UID: "eae75cde-0bd4-43af-9228-a043514f9f29"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:53:25.012984 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.012956 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eae75cde-0bd4-43af-9228-a043514f9f29-kube-api-access-kb6pm" (OuterVolumeSpecName: "kube-api-access-kb6pm") pod "eae75cde-0bd4-43af-9228-a043514f9f29" (UID: "eae75cde-0bd4-43af-9228-a043514f9f29"). InnerVolumeSpecName "kube-api-access-kb6pm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:53:25.016995 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.016967 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-util" (OuterVolumeSpecName: "util") pod "eae75cde-0bd4-43af-9228-a043514f9f29" (UID: "eae75cde-0bd4-43af-9228-a043514f9f29"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:53:25.112414 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.112320 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:53:25.112414 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.112354 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb6pm\" (UniqueName: \"kubernetes.io/projected/eae75cde-0bd4-43af-9228-a043514f9f29-kube-api-access-kb6pm\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:53:25.112414 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.112366 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae75cde-0bd4-43af-9228-a043514f9f29-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:53:25.863003 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.862972 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" event={"ID":"eae75cde-0bd4-43af-9228-a043514f9f29","Type":"ContainerDied","Data":"9d599996954454631c1c9a924ba1bf209e721dd76b5bd435527ee3f29e309bb3"} Jun 03 13:53:25.863003 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.863003 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d599996954454631c1c9a924ba1bf209e721dd76b5bd435527ee3f29e309bb3" Jun 03 13:53:25.863243 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:25.863043 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdcr8lj" Jun 03 13:53:33.568576 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.568542 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf"] Jun 03 13:53:33.569005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.568847 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eae75cde-0bd4-43af-9228-a043514f9f29" containerName="pull" Jun 03 13:53:33.569005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.568861 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae75cde-0bd4-43af-9228-a043514f9f29" containerName="pull" Jun 03 13:53:33.569005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.568867 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eae75cde-0bd4-43af-9228-a043514f9f29" containerName="extract" Jun 03 13:53:33.569005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.568873 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae75cde-0bd4-43af-9228-a043514f9f29" containerName="extract" Jun 03 13:53:33.569005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.568899 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eae75cde-0bd4-43af-9228-a043514f9f29" containerName="util" Jun 03 13:53:33.569005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.568906 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae75cde-0bd4-43af-9228-a043514f9f29" containerName="util" Jun 03 13:53:33.569005 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.568954 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="eae75cde-0bd4-43af-9228-a043514f9f29" containerName="extract" Jun 03 13:53:33.573820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.573804 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.580087 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.580065 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 03 13:53:33.580228 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.580164 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-trdzv\"" Jun 03 13:53:33.580321 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.580164 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 03 13:53:33.581446 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.581429 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 03 13:53:33.583395 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.583378 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 03 13:53:33.604636 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.604613 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf"] Jun 03 13:53:33.675071 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.675043 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebc44fb7-a947-4b67-8f6d-1f17a9577814-webhook-cert\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.675167 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.675111 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebc44fb7-a947-4b67-8f6d-1f17a9577814-manifests\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.675167 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.675131 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqq55\" (UniqueName: \"kubernetes.io/projected/ebc44fb7-a947-4b67-8f6d-1f17a9577814-kube-api-access-hqq55\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.675167 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.675148 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebc44fb7-a947-4b67-8f6d-1f17a9577814-tmp\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.675167 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.675162 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebc44fb7-a947-4b67-8f6d-1f17a9577814-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.776467 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.776430 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebc44fb7-a947-4b67-8f6d-1f17a9577814-webhook-cert\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.776583 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.776478 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebc44fb7-a947-4b67-8f6d-1f17a9577814-manifests\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.776583 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.776498 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hqq55\" (UniqueName: \"kubernetes.io/projected/ebc44fb7-a947-4b67-8f6d-1f17a9577814-kube-api-access-hqq55\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.776583 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.776514 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebc44fb7-a947-4b67-8f6d-1f17a9577814-tmp\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.776583 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.776534 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebc44fb7-a947-4b67-8f6d-1f17a9577814-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.776812 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.776789 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebc44fb7-a947-4b67-8f6d-1f17a9577814-manifests\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.778706 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.778685 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebc44fb7-a947-4b67-8f6d-1f17a9577814-tmp\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.778955 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.778930 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebc44fb7-a947-4b67-8f6d-1f17a9577814-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.779067 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.779049 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebc44fb7-a947-4b67-8f6d-1f17a9577814-webhook-cert\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.787376 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.787353 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqq55\" (UniqueName: \"kubernetes.io/projected/ebc44fb7-a947-4b67-8f6d-1f17a9577814-kube-api-access-hqq55\") pod \"opendatahub-operator-controller-manager-5496d4666c-dlkbf\" (UID: \"ebc44fb7-a947-4b67-8f6d-1f17a9577814\") " pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:33.883794 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:33.883700 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:34.029825 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:34.029805 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf"] Jun 03 13:53:34.032127 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:53:34.032097 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebc44fb7_a947_4b67_8f6d_1f17a9577814.slice/crio-c23bb49951609d679fff92e292bdddcea2cbdefb569b429573528ce6d2909b9f WatchSource:0}: Error finding container c23bb49951609d679fff92e292bdddcea2cbdefb569b429573528ce6d2909b9f: Status 404 returned error can't find the container with id c23bb49951609d679fff92e292bdddcea2cbdefb569b429573528ce6d2909b9f Jun 03 13:53:34.891363 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:34.891325 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" event={"ID":"ebc44fb7-a947-4b67-8f6d-1f17a9577814","Type":"ContainerStarted","Data":"c23bb49951609d679fff92e292bdddcea2cbdefb569b429573528ce6d2909b9f"} Jun 03 13:53:37.902058 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:37.902020 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" event={"ID":"ebc44fb7-a947-4b67-8f6d-1f17a9577814","Type":"ContainerStarted","Data":"1bc51f25bc60c6af7403504af1d8a9d6b85b77a0e8411bcc51253d65d051ad42"} Jun 03 13:53:38.906043 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:38.906011 2573 generic.go:358] "Generic (PLEG): container finished" podID="ebc44fb7-a947-4b67-8f6d-1f17a9577814" containerID="1bc51f25bc60c6af7403504af1d8a9d6b85b77a0e8411bcc51253d65d051ad42" exitCode=0 Jun 03 13:53:38.906472 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:38.906096 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" event={"ID":"ebc44fb7-a947-4b67-8f6d-1f17a9577814","Type":"ContainerDied","Data":"1bc51f25bc60c6af7403504af1d8a9d6b85b77a0e8411bcc51253d65d051ad42"} Jun 03 13:53:39.910090 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:39.910058 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" event={"ID":"ebc44fb7-a947-4b67-8f6d-1f17a9577814","Type":"ContainerStarted","Data":"7d92c27be0949e3c3c3b73998f261fcc29a94f9626066f67991eb60319c57466"} Jun 03 13:53:39.910449 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:39.910205 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:39.936809 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:39.936756 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" podStartSLOduration=1.622662449 podStartE2EDuration="6.936718986s" podCreationTimestamp="2026-06-03 13:53:33 +0000 UTC" firstStartedPulling="2026-06-03 13:53:34.033747091 +0000 UTC m=+371.804435797" lastFinishedPulling="2026-06-03 13:53:39.347803628 +0000 UTC m=+377.118492334" observedRunningTime="2026-06-03 13:53:39.934749406 +0000 UTC m=+377.705438142" watchObservedRunningTime="2026-06-03 13:53:39.936718986 +0000 UTC m=+377.707407712" Jun 03 13:53:50.915073 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:50.915045 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5496d4666c-dlkbf" Jun 03 13:53:53.327719 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.327684 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t"] Jun 03 13:53:53.335633 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.335612 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.340545 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.340523 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 13:53:53.340798 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.340782 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 13:53:53.340868 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.340795 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vf5vf\"" Jun 03 13:53:53.350509 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.350488 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t"] Jun 03 13:53:53.419098 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.419063 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cr4d\" (UniqueName: \"kubernetes.io/projected/7c7965af-095c-4e83-ac7a-b66b0cd4425c-kube-api-access-5cr4d\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.419258 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.419115 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.419258 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.419159 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.471263 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.471238 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g"] Jun 03 13:53:53.474377 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.474363 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.479492 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.479468 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 03 13:53:53.479608 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.479467 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 13:53:53.479788 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.479767 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 13:53:53.479954 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.479937 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kr2np\"" Jun 03 13:53:53.481896 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.481864 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 03 13:53:53.495280 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.495259 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g"] Jun 03 13:53:53.520493 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.520471 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5cr4d\" (UniqueName: \"kubernetes.io/projected/7c7965af-095c-4e83-ac7a-b66b0cd4425c-kube-api-access-5cr4d\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.520588 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.520514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkqdm\" (UniqueName: \"kubernetes.io/projected/ac1e6c67-5211-4357-92ca-261541bcb148-kube-api-access-bkqdm\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.520588 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.520535 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.520675 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.520600 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.520727 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.520669 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac1e6c67-5211-4357-92ca-261541bcb148-tmp\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.520727 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.520704 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac1e6c67-5211-4357-92ca-261541bcb148-tls-certs\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.520969 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.520948 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.521036 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.520969 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.539978 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.539952 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cr4d\" (UniqueName: \"kubernetes.io/projected/7c7965af-095c-4e83-ac7a-b66b0cd4425c-kube-api-access-5cr4d\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.621308 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.621250 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bkqdm\" (UniqueName: \"kubernetes.io/projected/ac1e6c67-5211-4357-92ca-261541bcb148-kube-api-access-bkqdm\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.621308 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.621293 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac1e6c67-5211-4357-92ca-261541bcb148-tmp\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.621449 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.621320 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac1e6c67-5211-4357-92ca-261541bcb148-tls-certs\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.623334 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.623314 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac1e6c67-5211-4357-92ca-261541bcb148-tmp\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.623557 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.623537 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac1e6c67-5211-4357-92ca-261541bcb148-tls-certs\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.644789 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.644765 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:53:53.653047 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.653025 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkqdm\" (UniqueName: \"kubernetes.io/projected/ac1e6c67-5211-4357-92ca-261541bcb148-kube-api-access-bkqdm\") pod \"kube-auth-proxy-854c9b8647-m9j5g\" (UID: \"ac1e6c67-5211-4357-92ca-261541bcb148\") " pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.753730 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.753701 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-bb9mp"] Jun 03 13:53:53.758553 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.758536 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:53.763031 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.763012 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6xw4g\"" Jun 03 13:53:53.763287 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.763269 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 03 13:53:53.782855 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.782836 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-bb9mp"] Jun 03 13:53:53.783380 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.783361 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" Jun 03 13:53:53.823126 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.823091 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4smdf\" (UniqueName: \"kubernetes.io/projected/1f638895-30e1-4010-91d1-5f379b498f3b-kube-api-access-4smdf\") pod \"odh-model-controller-76588b95-bb9mp\" (UID: \"1f638895-30e1-4010-91d1-5f379b498f3b\") " pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:53.823304 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.823184 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f638895-30e1-4010-91d1-5f379b498f3b-cert\") pod \"odh-model-controller-76588b95-bb9mp\" (UID: \"1f638895-30e1-4010-91d1-5f379b498f3b\") " pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:53.833314 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.833252 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t"] Jun 03 13:53:53.924254 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.924222 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4smdf\" (UniqueName: \"kubernetes.io/projected/1f638895-30e1-4010-91d1-5f379b498f3b-kube-api-access-4smdf\") pod \"odh-model-controller-76588b95-bb9mp\" (UID: \"1f638895-30e1-4010-91d1-5f379b498f3b\") " pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:53.924421 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.924283 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f638895-30e1-4010-91d1-5f379b498f3b-cert\") pod \"odh-model-controller-76588b95-bb9mp\" (UID: \"1f638895-30e1-4010-91d1-5f379b498f3b\") " pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:53.924421 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:53:53.924391 2573 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 03 13:53:53.924538 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:53:53.924465 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f638895-30e1-4010-91d1-5f379b498f3b-cert podName:1f638895-30e1-4010-91d1-5f379b498f3b nodeName:}" failed. No retries permitted until 2026-06-03 13:53:54.424441233 +0000 UTC m=+392.195129950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f638895-30e1-4010-91d1-5f379b498f3b-cert") pod "odh-model-controller-76588b95-bb9mp" (UID: "1f638895-30e1-4010-91d1-5f379b498f3b") : secret "odh-model-controller-webhook-cert" not found Jun 03 13:53:53.944831 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.942947 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g"] Jun 03 13:53:53.948019 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.947999 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4smdf\" (UniqueName: \"kubernetes.io/projected/1f638895-30e1-4010-91d1-5f379b498f3b-kube-api-access-4smdf\") pod \"odh-model-controller-76588b95-bb9mp\" (UID: \"1f638895-30e1-4010-91d1-5f379b498f3b\") " pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:53.954610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.954583 2573 generic.go:358] "Generic (PLEG): container finished" podID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerID="456076ac39f8b910257e3688149747eca8dfad07c35cf03147e4f3a7148fc1b7" exitCode=0 Jun 03 13:53:53.954720 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.954659 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" event={"ID":"7c7965af-095c-4e83-ac7a-b66b0cd4425c","Type":"ContainerDied","Data":"456076ac39f8b910257e3688149747eca8dfad07c35cf03147e4f3a7148fc1b7"} Jun 03 13:53:53.954720 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:53.954691 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" event={"ID":"7c7965af-095c-4e83-ac7a-b66b0cd4425c","Type":"ContainerStarted","Data":"f38fd83ad425a1f4dfa8a3904b65f4bd13fee36f00bf127604632240c464e6ad"} Jun 03 13:53:53.976587 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:53:53.976563 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1e6c67_5211_4357_92ca_261541bcb148.slice/crio-0636483408a45dd95a4f40d11e04f5fe82e1b75920abee450a530da468433f2e WatchSource:0}: Error finding container 0636483408a45dd95a4f40d11e04f5fe82e1b75920abee450a530da468433f2e: Status 404 returned error can't find the container with id 0636483408a45dd95a4f40d11e04f5fe82e1b75920abee450a530da468433f2e Jun 03 13:53:54.429045 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:54.428996 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f638895-30e1-4010-91d1-5f379b498f3b-cert\") pod \"odh-model-controller-76588b95-bb9mp\" (UID: \"1f638895-30e1-4010-91d1-5f379b498f3b\") " pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:54.431409 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:54.431385 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f638895-30e1-4010-91d1-5f379b498f3b-cert\") pod \"odh-model-controller-76588b95-bb9mp\" (UID: \"1f638895-30e1-4010-91d1-5f379b498f3b\") " pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:54.668425 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:54.668362 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:54.852606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:54.852573 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-bb9mp"] Jun 03 13:53:54.963770 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:54.963736 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-bb9mp" event={"ID":"1f638895-30e1-4010-91d1-5f379b498f3b","Type":"ContainerStarted","Data":"694723b66904db1bcc4a7720e3800444b2ccea728e287643f6b89247623c3512"} Jun 03 13:53:54.965626 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:54.965588 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" event={"ID":"ac1e6c67-5211-4357-92ca-261541bcb148","Type":"ContainerStarted","Data":"0636483408a45dd95a4f40d11e04f5fe82e1b75920abee450a530da468433f2e"} Jun 03 13:53:54.967712 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:54.967679 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" event={"ID":"7c7965af-095c-4e83-ac7a-b66b0cd4425c","Type":"ContainerStarted","Data":"62926940dcdc925cddd44240bb821efb430c5b89389c44c5733d244438c70114"} Jun 03 13:53:55.230931 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.230838 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-6577596cd5-2q2ft"] Jun 03 13:53:55.234388 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.234361 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:55.239112 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.239087 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 03 13:53:55.264878 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.264855 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-2q2ft"] Jun 03 13:53:55.338129 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.338066 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0652c0c-1462-446a-a77a-add5ae61b3a0-tls-certs\") pod \"model-serving-api-6577596cd5-2q2ft\" (UID: \"d0652c0c-1462-446a-a77a-add5ae61b3a0\") " pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:55.338298 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.338147 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8ztq\" (UniqueName: \"kubernetes.io/projected/d0652c0c-1462-446a-a77a-add5ae61b3a0-kube-api-access-q8ztq\") pod \"model-serving-api-6577596cd5-2q2ft\" (UID: \"d0652c0c-1462-446a-a77a-add5ae61b3a0\") " pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:55.438572 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.438534 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0652c0c-1462-446a-a77a-add5ae61b3a0-tls-certs\") pod \"model-serving-api-6577596cd5-2q2ft\" (UID: \"d0652c0c-1462-446a-a77a-add5ae61b3a0\") " pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:55.439065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.438622 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q8ztq\" (UniqueName: \"kubernetes.io/projected/d0652c0c-1462-446a-a77a-add5ae61b3a0-kube-api-access-q8ztq\") pod \"model-serving-api-6577596cd5-2q2ft\" (UID: \"d0652c0c-1462-446a-a77a-add5ae61b3a0\") " pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:55.439065 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:53:55.438745 2573 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 03 13:53:55.439065 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:53:55.438817 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0652c0c-1462-446a-a77a-add5ae61b3a0-tls-certs podName:d0652c0c-1462-446a-a77a-add5ae61b3a0 nodeName:}" failed. No retries permitted until 2026-06-03 13:53:55.938795243 +0000 UTC m=+393.709483972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d0652c0c-1462-446a-a77a-add5ae61b3a0-tls-certs") pod "model-serving-api-6577596cd5-2q2ft" (UID: "d0652c0c-1462-446a-a77a-add5ae61b3a0") : secret "model-serving-api-tls" not found Jun 03 13:53:55.455857 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.455808 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8ztq\" (UniqueName: \"kubernetes.io/projected/d0652c0c-1462-446a-a77a-add5ae61b3a0-kube-api-access-q8ztq\") pod \"model-serving-api-6577596cd5-2q2ft\" (UID: \"d0652c0c-1462-446a-a77a-add5ae61b3a0\") " pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:55.943998 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.943465 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0652c0c-1462-446a-a77a-add5ae61b3a0-tls-certs\") pod \"model-serving-api-6577596cd5-2q2ft\" (UID: \"d0652c0c-1462-446a-a77a-add5ae61b3a0\") " pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:55.946404 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.946334 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0652c0c-1462-446a-a77a-add5ae61b3a0-tls-certs\") pod \"model-serving-api-6577596cd5-2q2ft\" (UID: \"d0652c0c-1462-446a-a77a-add5ae61b3a0\") " pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:55.974166 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.974132 2573 generic.go:358] "Generic (PLEG): container finished" podID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerID="62926940dcdc925cddd44240bb821efb430c5b89389c44c5733d244438c70114" exitCode=0 Jun 03 13:53:55.974315 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:55.974224 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" event={"ID":"7c7965af-095c-4e83-ac7a-b66b0cd4425c","Type":"ContainerDied","Data":"62926940dcdc925cddd44240bb821efb430c5b89389c44c5733d244438c70114"} Jun 03 13:53:56.147559 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:56.147526 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:53:58.229806 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:58.229770 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-2q2ft"] Jun 03 13:53:58.232973 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:53:58.232945 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0652c0c_1462_446a_a77a_add5ae61b3a0.slice/crio-e0fae01ecf5b2b1581797c928b9d49ad0f02eff60ec8a54401faf4683e422834 WatchSource:0}: Error finding container e0fae01ecf5b2b1581797c928b9d49ad0f02eff60ec8a54401faf4683e422834: Status 404 returned error can't find the container with id e0fae01ecf5b2b1581797c928b9d49ad0f02eff60ec8a54401faf4683e422834 Jun 03 13:53:58.986213 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:58.986179 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-bb9mp" event={"ID":"1f638895-30e1-4010-91d1-5f379b498f3b","Type":"ContainerStarted","Data":"4f899df00cf679d90b76613c70b39ce44bcf1843a1efdb5f9e35b132a64681ba"} Jun 03 13:53:58.986394 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:58.986292 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:53:58.987335 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:58.987312 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-2q2ft" event={"ID":"d0652c0c-1462-446a-a77a-add5ae61b3a0","Type":"ContainerStarted","Data":"e0fae01ecf5b2b1581797c928b9d49ad0f02eff60ec8a54401faf4683e422834"} Jun 03 13:53:58.988728 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:58.988692 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" event={"ID":"ac1e6c67-5211-4357-92ca-261541bcb148","Type":"ContainerStarted","Data":"0bd159188ea9d5aaf110693834add28fc0c5eb6782656c92aee0fe1b202e504c"} Jun 03 13:53:58.990994 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:58.990961 2573 generic.go:358] "Generic (PLEG): container finished" podID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerID="27c0b7867ebbe66a9c909a8a9540de6b6ea5ba3cefbda3bef20b9fe1c7baa9d8" exitCode=0 Jun 03 13:53:58.991093 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:58.991064 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" event={"ID":"7c7965af-095c-4e83-ac7a-b66b0cd4425c","Type":"ContainerDied","Data":"27c0b7867ebbe66a9c909a8a9540de6b6ea5ba3cefbda3bef20b9fe1c7baa9d8"} Jun 03 13:53:59.003140 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:59.003093 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-bb9mp" podStartSLOduration=2.724283895 podStartE2EDuration="6.003079824s" podCreationTimestamp="2026-06-03 13:53:53 +0000 UTC" firstStartedPulling="2026-06-03 13:53:54.860777355 +0000 UTC m=+392.631466072" lastFinishedPulling="2026-06-03 13:53:58.139573285 +0000 UTC m=+395.910262001" observedRunningTime="2026-06-03 13:53:59.001328022 +0000 UTC m=+396.772016747" watchObservedRunningTime="2026-06-03 13:53:59.003079824 +0000 UTC m=+396.773768548" Jun 03 13:53:59.037007 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:59.036958 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-854c9b8647-m9j5g" podStartSLOduration=1.8907449189999999 podStartE2EDuration="6.036945323s" podCreationTimestamp="2026-06-03 13:53:53 +0000 UTC" firstStartedPulling="2026-06-03 13:53:53.978347076 +0000 UTC m=+391.749035779" lastFinishedPulling="2026-06-03 13:53:58.124547461 +0000 UTC m=+395.895236183" observedRunningTime="2026-06-03 13:53:59.03532187 +0000 UTC m=+396.806010596" watchObservedRunningTime="2026-06-03 13:53:59.036945323 +0000 UTC m=+396.807634048" Jun 03 13:53:59.996036 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:59.996001 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-2q2ft" event={"ID":"d0652c0c-1462-446a-a77a-add5ae61b3a0","Type":"ContainerStarted","Data":"e92c8c0d667273fed5d0a3d942a273a7fcffb251a6aa50954bf7766372d3bdf5"} Jun 03 13:53:59.996830 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:53:59.996694 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:54:00.021197 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.021139 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/model-serving-api-6577596cd5-2q2ft" podStartSLOduration=3.339948691 podStartE2EDuration="5.021120485s" podCreationTimestamp="2026-06-03 13:53:55 +0000 UTC" firstStartedPulling="2026-06-03 13:53:58.235229902 +0000 UTC m=+396.005918605" lastFinishedPulling="2026-06-03 13:53:59.916401695 +0000 UTC m=+397.687090399" observedRunningTime="2026-06-03 13:54:00.020062212 +0000 UTC m=+397.790750949" watchObservedRunningTime="2026-06-03 13:54:00.021120485 +0000 UTC m=+397.791809211" Jun 03 13:54:00.119510 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.119488 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:54:00.179853 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.179779 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-bundle\") pod \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " Jun 03 13:54:00.179853 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.179825 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cr4d\" (UniqueName: \"kubernetes.io/projected/7c7965af-095c-4e83-ac7a-b66b0cd4425c-kube-api-access-5cr4d\") pod \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " Jun 03 13:54:00.180059 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.179904 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-util\") pod \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\" (UID: \"7c7965af-095c-4e83-ac7a-b66b0cd4425c\") " Jun 03 13:54:00.180688 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.180661 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-bundle" (OuterVolumeSpecName: "bundle") pod "7c7965af-095c-4e83-ac7a-b66b0cd4425c" (UID: "7c7965af-095c-4e83-ac7a-b66b0cd4425c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:54:00.181945 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.181923 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c7965af-095c-4e83-ac7a-b66b0cd4425c-kube-api-access-5cr4d" (OuterVolumeSpecName: "kube-api-access-5cr4d") pod "7c7965af-095c-4e83-ac7a-b66b0cd4425c" (UID: "7c7965af-095c-4e83-ac7a-b66b0cd4425c"). InnerVolumeSpecName "kube-api-access-5cr4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:54:00.185050 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.185019 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-util" (OuterVolumeSpecName: "util") pod "7c7965af-095c-4e83-ac7a-b66b0cd4425c" (UID: "7c7965af-095c-4e83-ac7a-b66b0cd4425c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:54:00.280703 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.280669 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:54:00.280703 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.280696 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c7965af-095c-4e83-ac7a-b66b0cd4425c-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:54:00.280703 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:00.280709 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cr4d\" (UniqueName: \"kubernetes.io/projected/7c7965af-095c-4e83-ac7a-b66b0cd4425c-kube-api-access-5cr4d\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:54:01.001176 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:01.001105 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" Jun 03 13:54:01.001550 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:01.001104 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835ctz9t" event={"ID":"7c7965af-095c-4e83-ac7a-b66b0cd4425c","Type":"ContainerDied","Data":"f38fd83ad425a1f4dfa8a3904b65f4bd13fee36f00bf127604632240c464e6ad"} Jun 03 13:54:01.001550 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:01.001214 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f38fd83ad425a1f4dfa8a3904b65f4bd13fee36f00bf127604632240c464e6ad" Jun 03 13:54:04.776245 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.776214 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg"] Jun 03 13:54:04.776606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.776492 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerName="util" Jun 03 13:54:04.776606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.776503 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerName="util" Jun 03 13:54:04.776606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.776514 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerName="pull" Jun 03 13:54:04.776606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.776520 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerName="pull" Jun 03 13:54:04.776606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.776528 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerName="extract" Jun 03 13:54:04.776606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.776533 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerName="extract" Jun 03 13:54:04.776606 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.776577 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="7c7965af-095c-4e83-ac7a-b66b0cd4425c" containerName="extract" Jun 03 13:54:04.781355 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.781330 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:04.785235 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.785210 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 03 13:54:04.786458 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.786436 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qjgv4\"" Jun 03 13:54:04.807144 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.807110 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-522nb"] Jun 03 13:54:04.810453 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.810434 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:04.825587 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.825562 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-gvgnm\"" Jun 03 13:54:04.825836 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.825820 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 03 13:54:04.829702 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.829679 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg"] Jun 03 13:54:04.886899 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.886854 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-522nb"] Jun 03 13:54:04.918342 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.918312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pq5c\" (UniqueName: \"kubernetes.io/projected/beaec3fb-4774-4051-9de2-cdc52b5f9a73-kube-api-access-7pq5c\") pod \"llmisvc-controller-manager-c6b9847d7-522nb\" (UID: \"beaec3fb-4774-4051-9de2-cdc52b5f9a73\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:04.918491 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.918362 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c283a5e9-49ce-4120-84a9-0eb25761276a-cert\") pod \"kserve-controller-manager-7c99ffcdcd-qcbwg\" (UID: \"c283a5e9-49ce-4120-84a9-0eb25761276a\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:04.918491 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.918392 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/beaec3fb-4774-4051-9de2-cdc52b5f9a73-cert\") pod \"llmisvc-controller-manager-c6b9847d7-522nb\" (UID: \"beaec3fb-4774-4051-9de2-cdc52b5f9a73\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:04.918574 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:04.918486 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r66sx\" (UniqueName: \"kubernetes.io/projected/c283a5e9-49ce-4120-84a9-0eb25761276a-kube-api-access-r66sx\") pod \"kserve-controller-manager-7c99ffcdcd-qcbwg\" (UID: \"c283a5e9-49ce-4120-84a9-0eb25761276a\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:05.019014 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.018987 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r66sx\" (UniqueName: \"kubernetes.io/projected/c283a5e9-49ce-4120-84a9-0eb25761276a-kube-api-access-r66sx\") pod \"kserve-controller-manager-7c99ffcdcd-qcbwg\" (UID: \"c283a5e9-49ce-4120-84a9-0eb25761276a\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:05.019172 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.019021 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7pq5c\" (UniqueName: \"kubernetes.io/projected/beaec3fb-4774-4051-9de2-cdc52b5f9a73-kube-api-access-7pq5c\") pod \"llmisvc-controller-manager-c6b9847d7-522nb\" (UID: \"beaec3fb-4774-4051-9de2-cdc52b5f9a73\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:05.019172 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.019157 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c283a5e9-49ce-4120-84a9-0eb25761276a-cert\") pod \"kserve-controller-manager-7c99ffcdcd-qcbwg\" (UID: \"c283a5e9-49ce-4120-84a9-0eb25761276a\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:05.019305 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.019191 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/beaec3fb-4774-4051-9de2-cdc52b5f9a73-cert\") pod \"llmisvc-controller-manager-c6b9847d7-522nb\" (UID: \"beaec3fb-4774-4051-9de2-cdc52b5f9a73\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:05.021695 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.021672 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c283a5e9-49ce-4120-84a9-0eb25761276a-cert\") pod \"kserve-controller-manager-7c99ffcdcd-qcbwg\" (UID: \"c283a5e9-49ce-4120-84a9-0eb25761276a\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:05.021820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.021672 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/beaec3fb-4774-4051-9de2-cdc52b5f9a73-cert\") pod \"llmisvc-controller-manager-c6b9847d7-522nb\" (UID: \"beaec3fb-4774-4051-9de2-cdc52b5f9a73\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:05.046095 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.046028 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r66sx\" (UniqueName: \"kubernetes.io/projected/c283a5e9-49ce-4120-84a9-0eb25761276a-kube-api-access-r66sx\") pod \"kserve-controller-manager-7c99ffcdcd-qcbwg\" (UID: \"c283a5e9-49ce-4120-84a9-0eb25761276a\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:05.064687 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.064659 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pq5c\" (UniqueName: \"kubernetes.io/projected/beaec3fb-4774-4051-9de2-cdc52b5f9a73-kube-api-access-7pq5c\") pod \"llmisvc-controller-manager-c6b9847d7-522nb\" (UID: \"beaec3fb-4774-4051-9de2-cdc52b5f9a73\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:05.092963 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.092943 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:05.119804 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.119777 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:05.264097 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.264067 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg"] Jun 03 13:54:05.266326 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:54:05.266297 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc283a5e9_49ce_4120_84a9_0eb25761276a.slice/crio-5cf3aa4c6a0bea026897571b61826352a364d3a28af3378430916e581a8e1161 WatchSource:0}: Error finding container 5cf3aa4c6a0bea026897571b61826352a364d3a28af3378430916e581a8e1161: Status 404 returned error can't find the container with id 5cf3aa4c6a0bea026897571b61826352a364d3a28af3378430916e581a8e1161 Jun 03 13:54:05.299820 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:05.299666 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-522nb"] Jun 03 13:54:05.302094 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:54:05.302068 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbeaec3fb_4774_4051_9de2_cdc52b5f9a73.slice/crio-0199bb10be8ab249a079242a783f00476625f0361173bbb444fb23a22e2ee778 WatchSource:0}: Error finding container 0199bb10be8ab249a079242a783f00476625f0361173bbb444fb23a22e2ee778: Status 404 returned error can't find the container with id 0199bb10be8ab249a079242a783f00476625f0361173bbb444fb23a22e2ee778 Jun 03 13:54:06.017512 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:06.017479 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" event={"ID":"c283a5e9-49ce-4120-84a9-0eb25761276a","Type":"ContainerStarted","Data":"5cf3aa4c6a0bea026897571b61826352a364d3a28af3378430916e581a8e1161"} Jun 03 13:54:06.018441 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:06.018419 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" event={"ID":"beaec3fb-4774-4051-9de2-cdc52b5f9a73","Type":"ContainerStarted","Data":"0199bb10be8ab249a079242a783f00476625f0361173bbb444fb23a22e2ee778"} Jun 03 13:54:08.379477 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.379443 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4"] Jun 03 13:54:08.410183 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.410149 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.416331 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.416224 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vf5vf\"" Jun 03 13:54:08.416331 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.416284 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 13:54:08.416593 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.416474 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 13:54:08.416593 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.416503 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4"] Jun 03 13:54:08.549246 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.549208 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.549441 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.549260 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phhpf\" (UniqueName: \"kubernetes.io/projected/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-kube-api-access-phhpf\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.549441 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.549353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.650544 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.650510 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.650714 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.650599 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.650714 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.650629 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-phhpf\" (UniqueName: \"kubernetes.io/projected/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-kube-api-access-phhpf\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.650841 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.650820 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.650930 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.650908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.680067 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.680044 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phhpf\" (UniqueName: \"kubernetes.io/projected/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-kube-api-access-phhpf\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:08.723971 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:08.723949 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:09.833743 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:09.833655 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4"] Jun 03 13:54:09.998707 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:09.998676 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-bb9mp" Jun 03 13:54:10.287358 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:54:10.287329 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e70bff2_91a5_415d_88b4_7d6d8cf48bfd.slice/crio-f9d0d24d33fdf29c3170e0af4b5149acbe502df80ec8415e7a6c790d0d1090fa WatchSource:0}: Error finding container f9d0d24d33fdf29c3170e0af4b5149acbe502df80ec8415e7a6c790d0d1090fa: Status 404 returned error can't find the container with id f9d0d24d33fdf29c3170e0af4b5149acbe502df80ec8415e7a6c790d0d1090fa Jun 03 13:54:11.037506 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.037468 2573 generic.go:358] "Generic (PLEG): container finished" podID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerID="701be440cd4d0aeafcb52d1ea6178d3ab4a327079c25385a743a0741cd40bdf1" exitCode=0 Jun 03 13:54:11.038030 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.037549 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" event={"ID":"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd","Type":"ContainerDied","Data":"701be440cd4d0aeafcb52d1ea6178d3ab4a327079c25385a743a0741cd40bdf1"} Jun 03 13:54:11.038030 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.037582 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" event={"ID":"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd","Type":"ContainerStarted","Data":"f9d0d24d33fdf29c3170e0af4b5149acbe502df80ec8415e7a6c790d0d1090fa"} Jun 03 13:54:11.039118 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.039101 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" event={"ID":"c283a5e9-49ce-4120-84a9-0eb25761276a","Type":"ContainerStarted","Data":"6eef69e5678679998fcc47db0f2c52c22240e24a0dadde642c9240eb03c97d0b"} Jun 03 13:54:11.039218 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.039198 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:11.040579 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.040553 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" event={"ID":"beaec3fb-4774-4051-9de2-cdc52b5f9a73","Type":"ContainerStarted","Data":"1df42d45d953d81069abf90f7b062507b7313c88e3be6b594eed55e519801f15"} Jun 03 13:54:11.040710 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.040697 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:11.108721 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.108664 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" podStartSLOduration=1.848669111 podStartE2EDuration="7.108649189s" podCreationTimestamp="2026-06-03 13:54:04 +0000 UTC" firstStartedPulling="2026-06-03 13:54:05.30347264 +0000 UTC m=+403.074161343" lastFinishedPulling="2026-06-03 13:54:10.563452718 +0000 UTC m=+408.334141421" observedRunningTime="2026-06-03 13:54:11.106079606 +0000 UTC m=+408.876768332" watchObservedRunningTime="2026-06-03 13:54:11.108649189 +0000 UTC m=+408.879337913" Jun 03 13:54:11.159247 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:11.159193 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" podStartSLOduration=2.954791792 podStartE2EDuration="7.159178412s" podCreationTimestamp="2026-06-03 13:54:04 +0000 UTC" firstStartedPulling="2026-06-03 13:54:05.268112126 +0000 UTC m=+403.038800828" lastFinishedPulling="2026-06-03 13:54:09.47249874 +0000 UTC m=+407.243187448" observedRunningTime="2026-06-03 13:54:11.155769593 +0000 UTC m=+408.926458317" watchObservedRunningTime="2026-06-03 13:54:11.159178412 +0000 UTC m=+408.929867136" Jun 03 13:54:12.009253 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.009226 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-6577596cd5-2q2ft" Jun 03 13:54:12.482098 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.482063 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-d48bdd865-m5p2x"] Jun 03 13:54:12.484189 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.484172 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" Jun 03 13:54:12.492211 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.492192 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cvw9w\"" Jun 03 13:54:12.492334 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.492316 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 03 13:54:12.516037 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.516005 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-d48bdd865-m5p2x"] Jun 03 13:54:12.585233 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.585200 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9b8w\" (UniqueName: \"kubernetes.io/projected/a3a85c26-b143-49d0-b78a-d02bf1c8bec6-kube-api-access-j9b8w\") pod \"dashboard-redirect-d48bdd865-m5p2x\" (UID: \"a3a85c26-b143-49d0-b78a-d02bf1c8bec6\") " pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" Jun 03 13:54:12.585448 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.585246 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a3a85c26-b143-49d0-b78a-d02bf1c8bec6-redirect-config\") pod \"dashboard-redirect-d48bdd865-m5p2x\" (UID: \"a3a85c26-b143-49d0-b78a-d02bf1c8bec6\") " pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" Jun 03 13:54:12.686091 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.686056 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9b8w\" (UniqueName: \"kubernetes.io/projected/a3a85c26-b143-49d0-b78a-d02bf1c8bec6-kube-api-access-j9b8w\") pod \"dashboard-redirect-d48bdd865-m5p2x\" (UID: \"a3a85c26-b143-49d0-b78a-d02bf1c8bec6\") " pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" Jun 03 13:54:12.686091 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.686092 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a3a85c26-b143-49d0-b78a-d02bf1c8bec6-redirect-config\") pod \"dashboard-redirect-d48bdd865-m5p2x\" (UID: \"a3a85c26-b143-49d0-b78a-d02bf1c8bec6\") " pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" Jun 03 13:54:12.687050 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.687027 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a3a85c26-b143-49d0-b78a-d02bf1c8bec6-redirect-config\") pod \"dashboard-redirect-d48bdd865-m5p2x\" (UID: \"a3a85c26-b143-49d0-b78a-d02bf1c8bec6\") " pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" Jun 03 13:54:12.716523 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.716489 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9b8w\" (UniqueName: \"kubernetes.io/projected/a3a85c26-b143-49d0-b78a-d02bf1c8bec6-kube-api-access-j9b8w\") pod \"dashboard-redirect-d48bdd865-m5p2x\" (UID: \"a3a85c26-b143-49d0-b78a-d02bf1c8bec6\") " pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" Jun 03 13:54:12.793829 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.793759 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" Jun 03 13:54:12.950233 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:12.950013 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-d48bdd865-m5p2x"] Jun 03 13:54:12.952467 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:54:12.952439 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a85c26_b143_49d0_b78a_d02bf1c8bec6.slice/crio-2bb1ef575d0d799c6f2eb87c3871cefa15c497a0bfad1230a7f1b4ecd65628bf WatchSource:0}: Error finding container 2bb1ef575d0d799c6f2eb87c3871cefa15c497a0bfad1230a7f1b4ecd65628bf: Status 404 returned error can't find the container with id 2bb1ef575d0d799c6f2eb87c3871cefa15c497a0bfad1230a7f1b4ecd65628bf Jun 03 13:54:13.049076 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:13.048987 2573 generic.go:358] "Generic (PLEG): container finished" podID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerID="36b8b42484f52cf3e537651dfe3b9fb089c0836d0ba0b150379fe7572f7cd28d" exitCode=0 Jun 03 13:54:13.049234 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:13.049074 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" event={"ID":"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd","Type":"ContainerDied","Data":"36b8b42484f52cf3e537651dfe3b9fb089c0836d0ba0b150379fe7572f7cd28d"} Jun 03 13:54:13.050268 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:13.050232 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" event={"ID":"a3a85c26-b143-49d0-b78a-d02bf1c8bec6","Type":"ContainerStarted","Data":"2bb1ef575d0d799c6f2eb87c3871cefa15c497a0bfad1230a7f1b4ecd65628bf"} Jun 03 13:54:14.055927 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.055871 2573 generic.go:358] "Generic (PLEG): container finished" podID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerID="e7b6fcdfd6ff6f3cbd1ed948cc094bc5087d69de02a6a1827bbfc956b324a6d8" exitCode=0 Jun 03 13:54:14.056295 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.055953 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" event={"ID":"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd","Type":"ContainerDied","Data":"e7b6fcdfd6ff6f3cbd1ed948cc094bc5087d69de02a6a1827bbfc956b324a6d8"} Jun 03 13:54:14.452632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.452596 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb"] Jun 03 13:54:14.454973 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.454956 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:14.468625 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.468606 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-87bd4\"" Jun 03 13:54:14.468625 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.468615 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 03 13:54:14.469586 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.469571 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 03 13:54:14.514849 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.514820 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb"] Jun 03 13:54:14.604406 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.604375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b255j\" (UniqueName: \"kubernetes.io/projected/3c184811-27c0-4ff9-8063-d729ccd651f6-kube-api-access-b255j\") pod \"servicemesh-operator3-55f49c5f94-xd2cb\" (UID: \"3c184811-27c0-4ff9-8063-d729ccd651f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:14.604571 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.604413 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/3c184811-27c0-4ff9-8063-d729ccd651f6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xd2cb\" (UID: \"3c184811-27c0-4ff9-8063-d729ccd651f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:14.704913 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.704816 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b255j\" (UniqueName: \"kubernetes.io/projected/3c184811-27c0-4ff9-8063-d729ccd651f6-kube-api-access-b255j\") pod \"servicemesh-operator3-55f49c5f94-xd2cb\" (UID: \"3c184811-27c0-4ff9-8063-d729ccd651f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:14.704913 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.704904 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/3c184811-27c0-4ff9-8063-d729ccd651f6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xd2cb\" (UID: \"3c184811-27c0-4ff9-8063-d729ccd651f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:14.708438 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.708409 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/3c184811-27c0-4ff9-8063-d729ccd651f6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xd2cb\" (UID: \"3c184811-27c0-4ff9-8063-d729ccd651f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:14.752632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.751854 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b255j\" (UniqueName: \"kubernetes.io/projected/3c184811-27c0-4ff9-8063-d729ccd651f6-kube-api-access-b255j\") pod \"servicemesh-operator3-55f49c5f94-xd2cb\" (UID: \"3c184811-27c0-4ff9-8063-d729ccd651f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:14.763711 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.763364 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:14.968801 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:14.968731 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb"] Jun 03 13:54:14.972953 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:54:14.972875 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c184811_27c0_4ff9_8063_d729ccd651f6.slice/crio-8850e61a7e40eb54dc00b1ce59629e059a23ab853a38b453ce7d6a46e423697d WatchSource:0}: Error finding container 8850e61a7e40eb54dc00b1ce59629e059a23ab853a38b453ce7d6a46e423697d: Status 404 returned error can't find the container with id 8850e61a7e40eb54dc00b1ce59629e059a23ab853a38b453ce7d6a46e423697d Jun 03 13:54:15.063299 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.063246 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" event={"ID":"3c184811-27c0-4ff9-8063-d729ccd651f6","Type":"ContainerStarted","Data":"8850e61a7e40eb54dc00b1ce59629e059a23ab853a38b453ce7d6a46e423697d"} Jun 03 13:54:15.212324 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.212300 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:15.309527 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.309440 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-util\") pod \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " Jun 03 13:54:15.309690 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.309528 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phhpf\" (UniqueName: \"kubernetes.io/projected/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-kube-api-access-phhpf\") pod \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " Jun 03 13:54:15.309690 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.309550 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-bundle\") pod \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\" (UID: \"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd\") " Jun 03 13:54:15.311069 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.311014 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-bundle" (OuterVolumeSpecName: "bundle") pod "9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" (UID: "9e70bff2-91a5-415d-88b4-7d6d8cf48bfd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:54:15.312111 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.312081 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-kube-api-access-phhpf" (OuterVolumeSpecName: "kube-api-access-phhpf") pod "9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" (UID: "9e70bff2-91a5-415d-88b4-7d6d8cf48bfd"). InnerVolumeSpecName "kube-api-access-phhpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:54:15.314910 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.314867 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-util" (OuterVolumeSpecName: "util") pod "9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" (UID: "9e70bff2-91a5-415d-88b4-7d6d8cf48bfd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:54:15.411246 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.411192 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:54:15.411246 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.411233 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phhpf\" (UniqueName: \"kubernetes.io/projected/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-kube-api-access-phhpf\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:54:15.411246 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:15.411251 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e70bff2-91a5-415d-88b4-7d6d8cf48bfd-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:54:16.068668 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:16.068596 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" Jun 03 13:54:16.069129 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:16.068597 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adfkd4" event={"ID":"9e70bff2-91a5-415d-88b4-7d6d8cf48bfd","Type":"ContainerDied","Data":"f9d0d24d33fdf29c3170e0af4b5149acbe502df80ec8415e7a6c790d0d1090fa"} Jun 03 13:54:16.069129 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:16.068710 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9d0d24d33fdf29c3170e0af4b5149acbe502df80ec8415e7a6c790d0d1090fa" Jun 03 13:54:19.082866 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:19.082832 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" event={"ID":"a3a85c26-b143-49d0-b78a-d02bf1c8bec6","Type":"ContainerStarted","Data":"3922492ed41a02ab786b0edd99bfda1e1a4392ed23eaf077c04b766e2ffa60b4"} Jun 03 13:54:19.203264 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:19.203211 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-d48bdd865-m5p2x" podStartSLOduration=1.663884038 podStartE2EDuration="7.203195566s" podCreationTimestamp="2026-06-03 13:54:12 +0000 UTC" firstStartedPulling="2026-06-03 13:54:12.953717774 +0000 UTC m=+410.724406480" lastFinishedPulling="2026-06-03 13:54:18.4930293 +0000 UTC m=+416.263718008" observedRunningTime="2026-06-03 13:54:19.19676285 +0000 UTC m=+416.967451574" watchObservedRunningTime="2026-06-03 13:54:19.203195566 +0000 UTC m=+416.973884317" Jun 03 13:54:22.094465 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:22.094376 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" event={"ID":"3c184811-27c0-4ff9-8063-d729ccd651f6","Type":"ContainerStarted","Data":"8f6dac9a267f527d202542e954d13d693800d60e31c915a70842c3442881551d"} Jun 03 13:54:22.094807 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:22.094533 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:22.127797 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:22.127734 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" podStartSLOduration=1.375530957 podStartE2EDuration="8.127716506s" podCreationTimestamp="2026-06-03 13:54:14 +0000 UTC" firstStartedPulling="2026-06-03 13:54:14.97686792 +0000 UTC m=+412.747556624" lastFinishedPulling="2026-06-03 13:54:21.72905347 +0000 UTC m=+419.499742173" observedRunningTime="2026-06-03 13:54:22.124361989 +0000 UTC m=+419.895050715" watchObservedRunningTime="2026-06-03 13:54:22.127716506 +0000 UTC m=+419.898405232" Jun 03 13:54:33.099814 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:33.099784 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xd2cb" Jun 03 13:54:42.046121 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:42.046031 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-522nb" Jun 03 13:54:42.049397 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:42.049373 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-qcbwg" Jun 03 13:54:43.648752 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.648716 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk"] Jun 03 13:54:43.649209 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.649041 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerName="extract" Jun 03 13:54:43.649209 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.649052 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerName="extract" Jun 03 13:54:43.649209 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.649087 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerName="pull" Jun 03 13:54:43.649209 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.649093 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerName="pull" Jun 03 13:54:43.649209 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.649099 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerName="util" Jun 03 13:54:43.649209 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.649105 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerName="util" Jun 03 13:54:43.649209 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.649162 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="9e70bff2-91a5-415d-88b4-7d6d8cf48bfd" containerName="extract" Jun 03 13:54:43.653520 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.653500 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.655788 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.655764 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 03 13:54:43.655937 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.655765 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 03 13:54:43.655937 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.655794 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 03 13:54:43.655937 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.655832 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8clvk\"" Jun 03 13:54:43.656106 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.656096 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 03 13:54:43.664208 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.664184 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk"] Jun 03 13:54:43.733829 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.733806 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.733977 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.733847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-token\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.733977 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.733872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.733977 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.733932 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-cacerts\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.733977 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.733953 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq6l8\" (UniqueName: \"kubernetes.io/projected/e06ac003-0404-4470-915d-b9d0dc13ed92-kube-api-access-nq6l8\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.734110 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.733977 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e06ac003-0404-4470-915d-b9d0dc13ed92-local-certs\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.734110 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.734031 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.834809 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.834765 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.834809 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.834803 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-cacerts\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.834809 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.834820 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nq6l8\" (UniqueName: \"kubernetes.io/projected/e06ac003-0404-4470-915d-b9d0dc13ed92-kube-api-access-nq6l8\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.835135 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.834939 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e06ac003-0404-4470-915d-b9d0dc13ed92-local-certs\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.835135 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.834990 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.835135 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.835056 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.835135 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.835107 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-token\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.835718 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.835664 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.837335 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.837302 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.837447 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.837388 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.837541 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.837521 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e06ac003-0404-4470-915d-b9d0dc13ed92-local-certs\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.837691 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.837672 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e06ac003-0404-4470-915d-b9d0dc13ed92-cacerts\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.842879 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.842858 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e06ac003-0404-4470-915d-b9d0dc13ed92-istio-token\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.843169 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.843152 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq6l8\" (UniqueName: \"kubernetes.io/projected/e06ac003-0404-4470-915d-b9d0dc13ed92-kube-api-access-nq6l8\") pod \"istiod-openshift-gateway-798958bb55-wbsnk\" (UID: \"e06ac003-0404-4470-915d-b9d0dc13ed92\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:43.962912 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:43.962785 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:44.107491 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:44.107462 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk"] Jun 03 13:54:44.109334 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:54:44.109302 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode06ac003_0404_4470_915d_b9d0dc13ed92.slice/crio-355facdc269d4d8056737d9749b5a54db49a9aa78c37c4bf72bef67d4ae2c1b1 WatchSource:0}: Error finding container 355facdc269d4d8056737d9749b5a54db49a9aa78c37c4bf72bef67d4ae2c1b1: Status 404 returned error can't find the container with id 355facdc269d4d8056737d9749b5a54db49a9aa78c37c4bf72bef67d4ae2c1b1 Jun 03 13:54:44.173666 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:44.173632 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" event={"ID":"e06ac003-0404-4470-915d-b9d0dc13ed92","Type":"ContainerStarted","Data":"355facdc269d4d8056737d9749b5a54db49a9aa78c37c4bf72bef67d4ae2c1b1"} Jun 03 13:54:46.692232 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:46.692182 2573 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 03 13:54:46.692605 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:46.692300 2573 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 03 13:54:47.188814 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:47.188781 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" event={"ID":"e06ac003-0404-4470-915d-b9d0dc13ed92","Type":"ContainerStarted","Data":"60e5b92cfb6fe310ba463092be65f24bf7193dd1776ab634831a29b111110dd7"} Jun 03 13:54:47.189036 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:47.188974 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:54:47.190599 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:47.190572 2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-wbsnk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 03 13:54:47.190714 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:47.190618 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" podUID="e06ac003-0404-4470-915d-b9d0dc13ed92" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 13:54:47.256280 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:47.256237 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" podStartSLOduration=1.675590648 podStartE2EDuration="4.256223033s" podCreationTimestamp="2026-06-03 13:54:43 +0000 UTC" firstStartedPulling="2026-06-03 13:54:44.111315299 +0000 UTC m=+441.882004004" lastFinishedPulling="2026-06-03 13:54:46.691947682 +0000 UTC m=+444.462636389" observedRunningTime="2026-06-03 13:54:47.253935491 +0000 UTC m=+445.024624216" watchObservedRunningTime="2026-06-03 13:54:47.256223033 +0000 UTC m=+445.026911758" Jun 03 13:54:48.193270 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:54:48.193242 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-wbsnk" Jun 03 13:55:09.432023 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.431987 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb"] Jun 03 13:55:09.435400 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.435385 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.438066 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.438041 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 13:55:09.438212 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.438087 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 13:55:09.438212 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.438038 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-57sf8\"" Jun 03 13:55:09.444937 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.444917 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb"] Jun 03 13:55:09.542473 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.542435 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.542629 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.542480 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.542629 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.542507 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwg22\" (UniqueName: \"kubernetes.io/projected/96f6f681-a4c0-49cc-9020-98d39183580d-kube-api-access-gwg22\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.643465 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.643429 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.643465 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.643466 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.643652 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.643483 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg22\" (UniqueName: \"kubernetes.io/projected/96f6f681-a4c0-49cc-9020-98d39183580d-kube-api-access-gwg22\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.643934 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.643910 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.643987 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.643921 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.652241 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.652212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwg22\" (UniqueName: \"kubernetes.io/projected/96f6f681-a4c0-49cc-9020-98d39183580d-kube-api-access-gwg22\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.745383 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.745328 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:09.833963 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.833931 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z"] Jun 03 13:55:09.838444 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.838425 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:09.846631 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.846588 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z"] Jun 03 13:55:09.874867 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.874848 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb"] Jun 03 13:55:09.876610 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:55:09.876581 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f6f681_a4c0_49cc_9020_98d39183580d.slice/crio-ae52f2aeeedbf257806aa636687498e85bd5b8dded5268de51316ba78aff7bbe WatchSource:0}: Error finding container ae52f2aeeedbf257806aa636687498e85bd5b8dded5268de51316ba78aff7bbe: Status 404 returned error can't find the container with id ae52f2aeeedbf257806aa636687498e85bd5b8dded5268de51316ba78aff7bbe Jun 03 13:55:09.946376 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.946353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:09.946482 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.946385 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz84x\" (UniqueName: \"kubernetes.io/projected/af0351b6-5c0c-413d-ba43-970f487699be-kube-api-access-kz84x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:09.946482 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:09.946471 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:10.047692 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.047636 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:10.047781 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.047700 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:10.047781 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.047728 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kz84x\" (UniqueName: \"kubernetes.io/projected/af0351b6-5c0c-413d-ba43-970f487699be-kube-api-access-kz84x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:10.048032 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.048011 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:10.048069 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.048037 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:10.055879 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.055851 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz84x\" (UniqueName: \"kubernetes.io/projected/af0351b6-5c0c-413d-ba43-970f487699be-kube-api-access-kz84x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:10.150556 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.150532 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:10.267827 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.267792 2573 generic.go:358] "Generic (PLEG): container finished" podID="96f6f681-a4c0-49cc-9020-98d39183580d" containerID="ace6066d40ed3ac7a7dc64451e5b26fb344fceae2d07706cd90fa471bb41a93c" exitCode=0 Jun 03 13:55:10.268001 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.267852 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" event={"ID":"96f6f681-a4c0-49cc-9020-98d39183580d","Type":"ContainerDied","Data":"ace6066d40ed3ac7a7dc64451e5b26fb344fceae2d07706cd90fa471bb41a93c"} Jun 03 13:55:10.268001 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.267873 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" event={"ID":"96f6f681-a4c0-49cc-9020-98d39183580d","Type":"ContainerStarted","Data":"ae52f2aeeedbf257806aa636687498e85bd5b8dded5268de51316ba78aff7bbe"} Jun 03 13:55:10.280532 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.280510 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z"] Jun 03 13:55:10.282505 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:55:10.282482 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf0351b6_5c0c_413d_ba43_970f487699be.slice/crio-c3d913a4850bfa30835c56edd4ae0c51bd8c0ef08ad87356d88cdb9b8acb1ee0 WatchSource:0}: Error finding container c3d913a4850bfa30835c56edd4ae0c51bd8c0ef08ad87356d88cdb9b8acb1ee0: Status 404 returned error can't find the container with id c3d913a4850bfa30835c56edd4ae0c51bd8c0ef08ad87356d88cdb9b8acb1ee0 Jun 03 13:55:10.442024 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.441991 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf"] Jun 03 13:55:10.445407 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.445391 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.454632 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.454605 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf"] Jun 03 13:55:10.551065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.550994 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh247\" (UniqueName: \"kubernetes.io/projected/04b928fb-3c40-4bcd-907d-743420427f1c-kube-api-access-sh247\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.551065 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.551047 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.551217 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.551093 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.651543 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.651514 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sh247\" (UniqueName: \"kubernetes.io/projected/04b928fb-3c40-4bcd-907d-743420427f1c-kube-api-access-sh247\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.651681 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.651560 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.651681 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.651594 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.652037 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.652009 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.652137 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.652043 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.659989 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.659957 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh247\" (UniqueName: \"kubernetes.io/projected/04b928fb-3c40-4bcd-907d-743420427f1c-kube-api-access-sh247\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.754699 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.754674 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:10.878063 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:10.878035 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf"] Jun 03 13:55:10.879945 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:55:10.879915 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b928fb_3c40_4bcd_907d_743420427f1c.slice/crio-91f535f034f5c57d4cd43a2484a914bd9c0242e0a41bbfe8963eab9bd2139568 WatchSource:0}: Error finding container 91f535f034f5c57d4cd43a2484a914bd9c0242e0a41bbfe8963eab9bd2139568: Status 404 returned error can't find the container with id 91f535f034f5c57d4cd43a2484a914bd9c0242e0a41bbfe8963eab9bd2139568 Jun 03 13:55:11.033410 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.033383 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz"] Jun 03 13:55:11.036577 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.036558 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.045265 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.045243 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz"] Jun 03 13:55:11.156295 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.156273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.156380 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.156311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9zh9\" (UniqueName: \"kubernetes.io/projected/a993f6dd-bcb4-4125-8be9-6504e72151c2-kube-api-access-k9zh9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.156438 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.156378 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.257170 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.257142 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.257270 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.257181 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k9zh9\" (UniqueName: \"kubernetes.io/projected/a993f6dd-bcb4-4125-8be9-6504e72151c2-kube-api-access-k9zh9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.257270 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.257217 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.257537 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.257523 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.257572 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.257526 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.267066 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.267041 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9zh9\" (UniqueName: \"kubernetes.io/projected/a993f6dd-bcb4-4125-8be9-6504e72151c2-kube-api-access-k9zh9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.276994 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.276969 2573 generic.go:358] "Generic (PLEG): container finished" podID="af0351b6-5c0c-413d-ba43-970f487699be" containerID="0edc0616c48ff60b19bccda018a2aa56997605df7dbcb4f1cc7644e762f8af3d" exitCode=0 Jun 03 13:55:11.277086 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.277052 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" event={"ID":"af0351b6-5c0c-413d-ba43-970f487699be","Type":"ContainerDied","Data":"0edc0616c48ff60b19bccda018a2aa56997605df7dbcb4f1cc7644e762f8af3d"} Jun 03 13:55:11.277127 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.277089 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" event={"ID":"af0351b6-5c0c-413d-ba43-970f487699be","Type":"ContainerStarted","Data":"c3d913a4850bfa30835c56edd4ae0c51bd8c0ef08ad87356d88cdb9b8acb1ee0"} Jun 03 13:55:11.278699 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.278675 2573 generic.go:358] "Generic (PLEG): container finished" podID="96f6f681-a4c0-49cc-9020-98d39183580d" containerID="49262cba2c9f4c2fc8abff22debabe896467d9e0b6c78e98bec07bfea06c7511" exitCode=0 Jun 03 13:55:11.278803 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.278753 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" event={"ID":"96f6f681-a4c0-49cc-9020-98d39183580d","Type":"ContainerDied","Data":"49262cba2c9f4c2fc8abff22debabe896467d9e0b6c78e98bec07bfea06c7511"} Jun 03 13:55:11.280151 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.280132 2573 generic.go:358] "Generic (PLEG): container finished" podID="04b928fb-3c40-4bcd-907d-743420427f1c" containerID="bff6150345da0206320e9173bc93bc027e7d8cfe8c238da07b90135d10c0538e" exitCode=0 Jun 03 13:55:11.280243 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.280160 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" event={"ID":"04b928fb-3c40-4bcd-907d-743420427f1c","Type":"ContainerDied","Data":"bff6150345da0206320e9173bc93bc027e7d8cfe8c238da07b90135d10c0538e"} Jun 03 13:55:11.280243 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.280176 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" event={"ID":"04b928fb-3c40-4bcd-907d-743420427f1c","Type":"ContainerStarted","Data":"91f535f034f5c57d4cd43a2484a914bd9c0242e0a41bbfe8963eab9bd2139568"} Jun 03 13:55:11.346075 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.346056 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:11.477721 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:11.477696 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz"] Jun 03 13:55:11.479252 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:55:11.479227 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda993f6dd_bcb4_4125_8be9_6504e72151c2.slice/crio-e609fa0fdc2820c057e808ce988fe6c5d37c1581283671b9cecb6bb300f2bc9a WatchSource:0}: Error finding container e609fa0fdc2820c057e808ce988fe6c5d37c1581283671b9cecb6bb300f2bc9a: Status 404 returned error can't find the container with id e609fa0fdc2820c057e808ce988fe6c5d37c1581283671b9cecb6bb300f2bc9a Jun 03 13:55:12.285903 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.285854 2573 generic.go:358] "Generic (PLEG): container finished" podID="af0351b6-5c0c-413d-ba43-970f487699be" containerID="994916af827a6632bf21ffc4275805f45ae39c98cba3d1a06dcf34aec6ab1a5d" exitCode=0 Jun 03 13:55:12.286071 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.285917 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" event={"ID":"af0351b6-5c0c-413d-ba43-970f487699be","Type":"ContainerDied","Data":"994916af827a6632bf21ffc4275805f45ae39c98cba3d1a06dcf34aec6ab1a5d"} Jun 03 13:55:12.287418 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.287390 2573 generic.go:358] "Generic (PLEG): container finished" podID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerID="c9309fea38b52d659602e7323b0a56567f0e7cba7e08714b54b058a9322c5572" exitCode=0 Jun 03 13:55:12.287524 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.287444 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" event={"ID":"a993f6dd-bcb4-4125-8be9-6504e72151c2","Type":"ContainerDied","Data":"c9309fea38b52d659602e7323b0a56567f0e7cba7e08714b54b058a9322c5572"} Jun 03 13:55:12.287524 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.287478 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" event={"ID":"a993f6dd-bcb4-4125-8be9-6504e72151c2","Type":"ContainerStarted","Data":"e609fa0fdc2820c057e808ce988fe6c5d37c1581283671b9cecb6bb300f2bc9a"} Jun 03 13:55:12.289767 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.289747 2573 generic.go:358] "Generic (PLEG): container finished" podID="96f6f681-a4c0-49cc-9020-98d39183580d" containerID="9efa1c8f9fdc4e88b433be9b17063281d8dea9bab956961aeeb329dd89629ddc" exitCode=0 Jun 03 13:55:12.289856 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.289821 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" event={"ID":"96f6f681-a4c0-49cc-9020-98d39183580d","Type":"ContainerDied","Data":"9efa1c8f9fdc4e88b433be9b17063281d8dea9bab956961aeeb329dd89629ddc"} Jun 03 13:55:12.291475 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.291426 2573 generic.go:358] "Generic (PLEG): container finished" podID="04b928fb-3c40-4bcd-907d-743420427f1c" containerID="bd4a5f5aa48afa948cdbba639c4f09719db8ac163052550850faebee5a2db4ca" exitCode=0 Jun 03 13:55:12.291537 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:12.291516 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" event={"ID":"04b928fb-3c40-4bcd-907d-743420427f1c","Type":"ContainerDied","Data":"bd4a5f5aa48afa948cdbba639c4f09719db8ac163052550850faebee5a2db4ca"} Jun 03 13:55:13.303117 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.303033 2573 generic.go:358] "Generic (PLEG): container finished" podID="af0351b6-5c0c-413d-ba43-970f487699be" containerID="76f29dce940f7b409e214d39ff7bf1c8ff2b5c0c3343c51ae0ff10fc64c34072" exitCode=0 Jun 03 13:55:13.303493 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.303112 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" event={"ID":"af0351b6-5c0c-413d-ba43-970f487699be","Type":"ContainerDied","Data":"76f29dce940f7b409e214d39ff7bf1c8ff2b5c0c3343c51ae0ff10fc64c34072"} Jun 03 13:55:13.304580 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.304559 2573 generic.go:358] "Generic (PLEG): container finished" podID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerID="9205c432eb35a9582ce63734b675f0dba212e615fbc5b02623466d174be85bb0" exitCode=0 Jun 03 13:55:13.304693 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.304633 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" event={"ID":"a993f6dd-bcb4-4125-8be9-6504e72151c2","Type":"ContainerDied","Data":"9205c432eb35a9582ce63734b675f0dba212e615fbc5b02623466d174be85bb0"} Jun 03 13:55:13.306544 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.306475 2573 generic.go:358] "Generic (PLEG): container finished" podID="04b928fb-3c40-4bcd-907d-743420427f1c" containerID="dea06114a4df9e941157ea557244ba75c4b8341c4f898cfa90d1328219ac68be" exitCode=0 Jun 03 13:55:13.306544 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.306504 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" event={"ID":"04b928fb-3c40-4bcd-907d-743420427f1c","Type":"ContainerDied","Data":"dea06114a4df9e941157ea557244ba75c4b8341c4f898cfa90d1328219ac68be"} Jun 03 13:55:13.448394 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.448367 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:13.576868 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.576780 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwg22\" (UniqueName: \"kubernetes.io/projected/96f6f681-a4c0-49cc-9020-98d39183580d-kube-api-access-gwg22\") pod \"96f6f681-a4c0-49cc-9020-98d39183580d\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " Jun 03 13:55:13.576868 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.576826 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-util\") pod \"96f6f681-a4c0-49cc-9020-98d39183580d\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " Jun 03 13:55:13.577076 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.576905 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-bundle\") pod \"96f6f681-a4c0-49cc-9020-98d39183580d\" (UID: \"96f6f681-a4c0-49cc-9020-98d39183580d\") " Jun 03 13:55:13.577402 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.577375 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-bundle" (OuterVolumeSpecName: "bundle") pod "96f6f681-a4c0-49cc-9020-98d39183580d" (UID: "96f6f681-a4c0-49cc-9020-98d39183580d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:13.578976 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.578945 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96f6f681-a4c0-49cc-9020-98d39183580d-kube-api-access-gwg22" (OuterVolumeSpecName: "kube-api-access-gwg22") pod "96f6f681-a4c0-49cc-9020-98d39183580d" (UID: "96f6f681-a4c0-49cc-9020-98d39183580d"). InnerVolumeSpecName "kube-api-access-gwg22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:55:13.582220 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.582198 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-util" (OuterVolumeSpecName: "util") pod "96f6f681-a4c0-49cc-9020-98d39183580d" (UID: "96f6f681-a4c0-49cc-9020-98d39183580d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:13.678183 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.678152 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:13.678183 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.678180 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwg22\" (UniqueName: \"kubernetes.io/projected/96f6f681-a4c0-49cc-9020-98d39183580d-kube-api-access-gwg22\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:13.678335 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:13.678190 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96f6f681-a4c0-49cc-9020-98d39183580d-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:14.313563 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.313533 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" Jun 03 13:55:14.313563 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.313544 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb" event={"ID":"96f6f681-a4c0-49cc-9020-98d39183580d","Type":"ContainerDied","Data":"ae52f2aeeedbf257806aa636687498e85bd5b8dded5268de51316ba78aff7bbe"} Jun 03 13:55:14.314027 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.313581 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae52f2aeeedbf257806aa636687498e85bd5b8dded5268de51316ba78aff7bbe" Jun 03 13:55:14.315344 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.315319 2573 generic.go:358] "Generic (PLEG): container finished" podID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerID="3924559138b6e9f76e86ae26306f42037d7d656a05212a1aa56965ba788b4faa" exitCode=0 Jun 03 13:55:14.315457 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.315378 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" event={"ID":"a993f6dd-bcb4-4125-8be9-6504e72151c2","Type":"ContainerDied","Data":"3924559138b6e9f76e86ae26306f42037d7d656a05212a1aa56965ba788b4faa"} Jun 03 13:55:14.466178 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.466157 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:14.469302 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.469281 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:14.586379 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.586298 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-bundle\") pod \"04b928fb-3c40-4bcd-907d-743420427f1c\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " Jun 03 13:55:14.586553 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.586377 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-util\") pod \"af0351b6-5c0c-413d-ba43-970f487699be\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " Jun 03 13:55:14.586553 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.586414 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-util\") pod \"04b928fb-3c40-4bcd-907d-743420427f1c\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " Jun 03 13:55:14.586553 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.586442 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-bundle\") pod \"af0351b6-5c0c-413d-ba43-970f487699be\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " Jun 03 13:55:14.586553 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.586468 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh247\" (UniqueName: \"kubernetes.io/projected/04b928fb-3c40-4bcd-907d-743420427f1c-kube-api-access-sh247\") pod \"04b928fb-3c40-4bcd-907d-743420427f1c\" (UID: \"04b928fb-3c40-4bcd-907d-743420427f1c\") " Jun 03 13:55:14.586553 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.586513 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz84x\" (UniqueName: \"kubernetes.io/projected/af0351b6-5c0c-413d-ba43-970f487699be-kube-api-access-kz84x\") pod \"af0351b6-5c0c-413d-ba43-970f487699be\" (UID: \"af0351b6-5c0c-413d-ba43-970f487699be\") " Jun 03 13:55:14.586915 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.586867 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-bundle" (OuterVolumeSpecName: "bundle") pod "04b928fb-3c40-4bcd-907d-743420427f1c" (UID: "04b928fb-3c40-4bcd-907d-743420427f1c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:14.587072 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.587048 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-bundle" (OuterVolumeSpecName: "bundle") pod "af0351b6-5c0c-413d-ba43-970f487699be" (UID: "af0351b6-5c0c-413d-ba43-970f487699be"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:14.588770 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.588740 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b928fb-3c40-4bcd-907d-743420427f1c-kube-api-access-sh247" (OuterVolumeSpecName: "kube-api-access-sh247") pod "04b928fb-3c40-4bcd-907d-743420427f1c" (UID: "04b928fb-3c40-4bcd-907d-743420427f1c"). InnerVolumeSpecName "kube-api-access-sh247". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:55:14.589112 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.589090 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0351b6-5c0c-413d-ba43-970f487699be-kube-api-access-kz84x" (OuterVolumeSpecName: "kube-api-access-kz84x") pod "af0351b6-5c0c-413d-ba43-970f487699be" (UID: "af0351b6-5c0c-413d-ba43-970f487699be"). InnerVolumeSpecName "kube-api-access-kz84x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:55:14.591582 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.591559 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-util" (OuterVolumeSpecName: "util") pod "af0351b6-5c0c-413d-ba43-970f487699be" (UID: "af0351b6-5c0c-413d-ba43-970f487699be"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:14.592583 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.592562 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-util" (OuterVolumeSpecName: "util") pod "04b928fb-3c40-4bcd-907d-743420427f1c" (UID: "04b928fb-3c40-4bcd-907d-743420427f1c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:14.687327 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.687302 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:14.687327 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.687325 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh247\" (UniqueName: \"kubernetes.io/projected/04b928fb-3c40-4bcd-907d-743420427f1c-kube-api-access-sh247\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:14.687534 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.687336 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz84x\" (UniqueName: \"kubernetes.io/projected/af0351b6-5c0c-413d-ba43-970f487699be-kube-api-access-kz84x\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:14.687534 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.687345 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:14.687534 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.687353 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af0351b6-5c0c-413d-ba43-970f487699be-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:14.687534 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:14.687361 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04b928fb-3c40-4bcd-907d-743420427f1c-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:15.321145 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.321106 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" event={"ID":"04b928fb-3c40-4bcd-907d-743420427f1c","Type":"ContainerDied","Data":"91f535f034f5c57d4cd43a2484a914bd9c0242e0a41bbfe8963eab9bd2139568"} Jun 03 13:55:15.321145 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.321145 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91f535f034f5c57d4cd43a2484a914bd9c0242e0a41bbfe8963eab9bd2139568" Jun 03 13:55:15.321145 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.321147 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf" Jun 03 13:55:15.323031 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.322991 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" event={"ID":"af0351b6-5c0c-413d-ba43-970f487699be","Type":"ContainerDied","Data":"c3d913a4850bfa30835c56edd4ae0c51bd8c0ef08ad87356d88cdb9b8acb1ee0"} Jun 03 13:55:15.323031 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.323022 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3d913a4850bfa30835c56edd4ae0c51bd8c0ef08ad87356d88cdb9b8acb1ee0" Jun 03 13:55:15.323237 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.323050 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z" Jun 03 13:55:15.450249 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.450219 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:15.594067 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.593976 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9zh9\" (UniqueName: \"kubernetes.io/projected/a993f6dd-bcb4-4125-8be9-6504e72151c2-kube-api-access-k9zh9\") pod \"a993f6dd-bcb4-4125-8be9-6504e72151c2\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " Jun 03 13:55:15.594067 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.594049 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-bundle\") pod \"a993f6dd-bcb4-4125-8be9-6504e72151c2\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " Jun 03 13:55:15.594295 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.594181 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-util\") pod \"a993f6dd-bcb4-4125-8be9-6504e72151c2\" (UID: \"a993f6dd-bcb4-4125-8be9-6504e72151c2\") " Jun 03 13:55:15.594690 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.594665 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-bundle" (OuterVolumeSpecName: "bundle") pod "a993f6dd-bcb4-4125-8be9-6504e72151c2" (UID: "a993f6dd-bcb4-4125-8be9-6504e72151c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:15.596085 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.596064 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a993f6dd-bcb4-4125-8be9-6504e72151c2-kube-api-access-k9zh9" (OuterVolumeSpecName: "kube-api-access-k9zh9") pod "a993f6dd-bcb4-4125-8be9-6504e72151c2" (UID: "a993f6dd-bcb4-4125-8be9-6504e72151c2"). InnerVolumeSpecName "kube-api-access-k9zh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:55:15.599474 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.599452 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-util" (OuterVolumeSpecName: "util") pod "a993f6dd-bcb4-4125-8be9-6504e72151c2" (UID: "a993f6dd-bcb4-4125-8be9-6504e72151c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:15.695390 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.695349 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:15.695390 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.695385 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a993f6dd-bcb4-4125-8be9-6504e72151c2-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:15.695390 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:15.695395 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9zh9\" (UniqueName: \"kubernetes.io/projected/a993f6dd-bcb4-4125-8be9-6504e72151c2-kube-api-access-k9zh9\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:16.328617 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:16.328591 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" Jun 03 13:55:16.329004 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:16.328580 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz" event={"ID":"a993f6dd-bcb4-4125-8be9-6504e72151c2","Type":"ContainerDied","Data":"e609fa0fdc2820c057e808ce988fe6c5d37c1581283671b9cecb6bb300f2bc9a"} Jun 03 13:55:16.329004 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:16.328706 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e609fa0fdc2820c057e808ce988fe6c5d37c1581283671b9cecb6bb300f2bc9a" Jun 03 13:55:36.915802 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.915767 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg"] Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916100 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af0351b6-5c0c-413d-ba43-970f487699be" containerName="extract" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916112 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0351b6-5c0c-413d-ba43-970f487699be" containerName="extract" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916121 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f6f681-a4c0-49cc-9020-98d39183580d" containerName="pull" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916126 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f6f681-a4c0-49cc-9020-98d39183580d" containerName="pull" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916139 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af0351b6-5c0c-413d-ba43-970f487699be" containerName="util" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916146 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0351b6-5c0c-413d-ba43-970f487699be" containerName="util" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916153 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b928fb-3c40-4bcd-907d-743420427f1c" containerName="extract" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916158 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b928fb-3c40-4bcd-907d-743420427f1c" containerName="extract" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916165 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerName="util" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916170 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerName="util" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916179 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b928fb-3c40-4bcd-907d-743420427f1c" containerName="util" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916184 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b928fb-3c40-4bcd-907d-743420427f1c" containerName="util" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916189 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerName="pull" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916194 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerName="pull" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916202 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f6f681-a4c0-49cc-9020-98d39183580d" containerName="extract" Jun 03 13:55:36.916199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916207 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f6f681-a4c0-49cc-9020-98d39183580d" containerName="extract" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916215 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b928fb-3c40-4bcd-907d-743420427f1c" containerName="pull" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916219 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b928fb-3c40-4bcd-907d-743420427f1c" containerName="pull" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916225 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af0351b6-5c0c-413d-ba43-970f487699be" containerName="pull" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916230 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0351b6-5c0c-413d-ba43-970f487699be" containerName="pull" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916237 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerName="extract" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916242 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerName="extract" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916249 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f6f681-a4c0-49cc-9020-98d39183580d" containerName="util" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916254 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f6f681-a4c0-49cc-9020-98d39183580d" containerName="util" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916301 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="96f6f681-a4c0-49cc-9020-98d39183580d" containerName="extract" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916309 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="04b928fb-3c40-4bcd-907d-743420427f1c" containerName="extract" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916317 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="a993f6dd-bcb4-4125-8be9-6504e72151c2" containerName="extract" Jun 03 13:55:36.916658 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.916324 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="af0351b6-5c0c-413d-ba43-970f487699be" containerName="extract" Jun 03 13:55:36.925251 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.925227 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:36.928648 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.928622 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 13:55:36.928790 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.928725 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 13:55:36.928918 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.928874 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jcfwk\"" Jun 03 13:55:36.944416 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.944390 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg"] Jun 03 13:55:36.964372 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.964346 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/373391b4-2053-41bd-a480-4f956764366a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" (UID: \"373391b4-2053-41bd-a480-4f956764366a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:36.964372 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:36.964377 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbn6l\" (UniqueName: \"kubernetes.io/projected/373391b4-2053-41bd-a480-4f956764366a-kube-api-access-zbn6l\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" (UID: \"373391b4-2053-41bd-a480-4f956764366a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:37.064728 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:37.064693 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/373391b4-2053-41bd-a480-4f956764366a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" (UID: \"373391b4-2053-41bd-a480-4f956764366a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:37.064728 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:37.064728 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zbn6l\" (UniqueName: \"kubernetes.io/projected/373391b4-2053-41bd-a480-4f956764366a-kube-api-access-zbn6l\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" (UID: \"373391b4-2053-41bd-a480-4f956764366a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:37.065153 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:37.065131 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/373391b4-2053-41bd-a480-4f956764366a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" (UID: \"373391b4-2053-41bd-a480-4f956764366a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:37.084726 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:37.084698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbn6l\" (UniqueName: \"kubernetes.io/projected/373391b4-2053-41bd-a480-4f956764366a-kube-api-access-zbn6l\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" (UID: \"373391b4-2053-41bd-a480-4f956764366a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:37.235668 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:37.235568 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:37.372765 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:37.372737 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg"] Jun 03 13:55:37.373958 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:55:37.373931 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373391b4_2053_41bd_a480_4f956764366a.slice/crio-1ed177d24683823666e18885874891004a82d248ccfcd14aad2a74f7df75f253 WatchSource:0}: Error finding container 1ed177d24683823666e18885874891004a82d248ccfcd14aad2a74f7df75f253: Status 404 returned error can't find the container with id 1ed177d24683823666e18885874891004a82d248ccfcd14aad2a74f7df75f253 Jun 03 13:55:37.411369 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:37.411341 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" event={"ID":"373391b4-2053-41bd-a480-4f956764366a","Type":"ContainerStarted","Data":"1ed177d24683823666e18885874891004a82d248ccfcd14aad2a74f7df75f253"} Jun 03 13:55:43.437127 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:43.437088 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" event={"ID":"373391b4-2053-41bd-a480-4f956764366a","Type":"ContainerStarted","Data":"70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4"} Jun 03 13:55:43.437587 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:43.437153 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:43.458452 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:43.458396 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" podStartSLOduration=2.10550915 podStartE2EDuration="7.458381291s" podCreationTimestamp="2026-06-03 13:55:36 +0000 UTC" firstStartedPulling="2026-06-03 13:55:37.376233648 +0000 UTC m=+495.146922351" lastFinishedPulling="2026-06-03 13:55:42.729105775 +0000 UTC m=+500.499794492" observedRunningTime="2026-06-03 13:55:43.45503679 +0000 UTC m=+501.225725515" watchObservedRunningTime="2026-06-03 13:55:43.458381291 +0000 UTC m=+501.229070015" Jun 03 13:55:54.443671 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:54.443639 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:55.631188 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.631151 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks"] Jun 03 13:55:55.634090 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.634067 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:55.651711 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.651680 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks"] Jun 03 13:55:55.712742 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.712705 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf69s\" (UniqueName: \"kubernetes.io/projected/ae720cc1-e83b-4793-8520-7917b0d17da4-kube-api-access-lf69s\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" (UID: \"ae720cc1-e83b-4793-8520-7917b0d17da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:55.712968 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.712788 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae720cc1-e83b-4793-8520-7917b0d17da4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" (UID: \"ae720cc1-e83b-4793-8520-7917b0d17da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:55.777193 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.777155 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks"] Jun 03 13:55:55.777412 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:55:55.777391 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-lf69s], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" podUID="ae720cc1-e83b-4793-8520-7917b0d17da4" Jun 03 13:55:55.814242 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.814201 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lf69s\" (UniqueName: \"kubernetes.io/projected/ae720cc1-e83b-4793-8520-7917b0d17da4-kube-api-access-lf69s\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" (UID: \"ae720cc1-e83b-4793-8520-7917b0d17da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:55.814423 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.814285 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae720cc1-e83b-4793-8520-7917b0d17da4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" (UID: \"ae720cc1-e83b-4793-8520-7917b0d17da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:55.814626 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.814611 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae720cc1-e83b-4793-8520-7917b0d17da4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" (UID: \"ae720cc1-e83b-4793-8520-7917b0d17da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:55.822359 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.822330 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f"] Jun 03 13:55:55.824625 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.824609 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:55.830133 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.830110 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf69s\" (UniqueName: \"kubernetes.io/projected/ae720cc1-e83b-4793-8520-7917b0d17da4-kube-api-access-lf69s\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" (UID: \"ae720cc1-e83b-4793-8520-7917b0d17da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:55.851937 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.851911 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f"] Jun 03 13:55:55.915273 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.915244 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nvfk\" (UniqueName: \"kubernetes.io/projected/8239e875-5f09-49ff-bd12-af3686f2927a-kube-api-access-8nvfk\") pod \"kuadrant-operator-controller-manager-84b657d985-mss6f\" (UID: \"8239e875-5f09-49ff-bd12-af3686f2927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:55.915433 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:55.915276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8239e875-5f09-49ff-bd12-af3686f2927a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-mss6f\" (UID: \"8239e875-5f09-49ff-bd12-af3686f2927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.016602 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.016570 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8nvfk\" (UniqueName: \"kubernetes.io/projected/8239e875-5f09-49ff-bd12-af3686f2927a-kube-api-access-8nvfk\") pod \"kuadrant-operator-controller-manager-84b657d985-mss6f\" (UID: \"8239e875-5f09-49ff-bd12-af3686f2927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.016748 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.016607 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8239e875-5f09-49ff-bd12-af3686f2927a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-mss6f\" (UID: \"8239e875-5f09-49ff-bd12-af3686f2927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.017039 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.017018 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8239e875-5f09-49ff-bd12-af3686f2927a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-mss6f\" (UID: \"8239e875-5f09-49ff-bd12-af3686f2927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.024644 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.024622 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nvfk\" (UniqueName: \"kubernetes.io/projected/8239e875-5f09-49ff-bd12-af3686f2927a-kube-api-access-8nvfk\") pod \"kuadrant-operator-controller-manager-84b657d985-mss6f\" (UID: \"8239e875-5f09-49ff-bd12-af3686f2927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.142310 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.142278 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.204142 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.203959 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg"] Jun 03 13:55:56.204300 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.204239 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" podUID="373391b4-2053-41bd-a480-4f956764366a" containerName="manager" containerID="cri-o://70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4" gracePeriod=2 Jun 03 13:55:56.207875 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.207844 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg"] Jun 03 13:55:56.208950 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.208907 2573 status_manager.go:895] "Failed to get status for pod" podUID="373391b4-2053-41bd-a480-4f956764366a" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.219613 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.219560 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks"] Jun 03 13:55:56.228199 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.228158 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks"] Jun 03 13:55:56.231570 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.231546 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m"] Jun 03 13:55:56.232163 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.232146 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="373391b4-2053-41bd-a480-4f956764366a" containerName="manager" Jun 03 13:55:56.232214 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.232164 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="373391b4-2053-41bd-a480-4f956764366a" containerName="manager" Jun 03 13:55:56.232252 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.232239 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="373391b4-2053-41bd-a480-4f956764366a" containerName="manager" Jun 03 13:55:56.234482 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.234462 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:56.237653 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.237629 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f"] Jun 03 13:55:56.247571 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.247523 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f"] Jun 03 13:55:56.252124 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.252030 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m"] Jun 03 13:55:56.271015 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.270717 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl"] Jun 03 13:55:56.271579 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.271526 2573 status_manager.go:895] "Failed to get status for pod" podUID="373391b4-2053-41bd-a480-4f956764366a" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.273183 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.273164 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:56.275481 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.275457 2573 status_manager.go:895] "Failed to get status for pod" podUID="373391b4-2053-41bd-a480-4f956764366a" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.291986 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.291928 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl"] Jun 03 13:55:56.319324 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.319292 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2a06da9-0d50-455d-a17e-fea1687e33df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-6v9vl\" (UID: \"c2a06da9-0d50-455d-a17e-fea1687e33df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:56.319459 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.319375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw2vh\" (UniqueName: \"kubernetes.io/projected/c2a06da9-0d50-455d-a17e-fea1687e33df-kube-api-access-tw2vh\") pod \"kuadrant-operator-controller-manager-84b657d985-6v9vl\" (UID: \"c2a06da9-0d50-455d-a17e-fea1687e33df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:56.319459 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.319416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2fed7a7-a594-4690-a0cf-b23c601c15d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-h2z7m\" (UID: \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:56.319459 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.319446 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2hnr\" (UniqueName: \"kubernetes.io/projected/f2fed7a7-a594-4690-a0cf-b23c601c15d8-kube-api-access-w2hnr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-h2z7m\" (UID: \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:56.420684 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.420654 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2a06da9-0d50-455d-a17e-fea1687e33df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-6v9vl\" (UID: \"c2a06da9-0d50-455d-a17e-fea1687e33df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:56.420805 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.420709 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tw2vh\" (UniqueName: \"kubernetes.io/projected/c2a06da9-0d50-455d-a17e-fea1687e33df-kube-api-access-tw2vh\") pod \"kuadrant-operator-controller-manager-84b657d985-6v9vl\" (UID: \"c2a06da9-0d50-455d-a17e-fea1687e33df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:56.420805 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.420738 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2fed7a7-a594-4690-a0cf-b23c601c15d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-h2z7m\" (UID: \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:56.420805 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.420766 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hnr\" (UniqueName: \"kubernetes.io/projected/f2fed7a7-a594-4690-a0cf-b23c601c15d8-kube-api-access-w2hnr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-h2z7m\" (UID: \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:56.421130 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.421102 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2a06da9-0d50-455d-a17e-fea1687e33df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-6v9vl\" (UID: \"c2a06da9-0d50-455d-a17e-fea1687e33df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:56.421237 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.421138 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2fed7a7-a594-4690-a0cf-b23c601c15d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-h2z7m\" (UID: \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:56.423915 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:55:56.423875 2573 log.go:32] "RunPodSandbox from runtime service failed" err=< Jun 03 13:55:56.423915 ip-10-0-143-102 kubenswrapper[2573]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-84b657d985-mss6f_kuadrant-system_8239e875-5f09-49ff-bd12-af3686f2927a_0(bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a): error adding pod kuadrant-system_kuadrant-operator-controller-manager-84b657d985-mss6f to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a" Netns:"/var/run/netns/73bbadfb-851b-438b-8e32-ceece23bc917" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-84b657d985-mss6f;K8S_POD_INFRA_CONTAINER_ID=bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a;K8S_POD_UID=8239e875-5f09-49ff-bd12-af3686f2927a" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f] networking: [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f/8239e875-5f09-49ff-bd12-af3686f2927a:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a network default NAD default] [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a network default NAD default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:27 [10.133.0.39/23] Jun 03 13:55:56.423915 ip-10-0-143-102 kubenswrapper[2573]: ' Jun 03 13:55:56.423915 ip-10-0-143-102 kubenswrapper[2573]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jun 03 13:55:56.423915 ip-10-0-143-102 kubenswrapper[2573]: > Jun 03 13:55:56.424092 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:55:56.423941 2573 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err=< Jun 03 13:55:56.424092 ip-10-0-143-102 kubenswrapper[2573]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-84b657d985-mss6f_kuadrant-system_8239e875-5f09-49ff-bd12-af3686f2927a_0(bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a): error adding pod kuadrant-system_kuadrant-operator-controller-manager-84b657d985-mss6f to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a" Netns:"/var/run/netns/73bbadfb-851b-438b-8e32-ceece23bc917" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-84b657d985-mss6f;K8S_POD_INFRA_CONTAINER_ID=bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a;K8S_POD_UID=8239e875-5f09-49ff-bd12-af3686f2927a" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f] networking: [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f/8239e875-5f09-49ff-bd12-af3686f2927a:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a network default NAD default] [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f bc712613ac22c3b2161a20d788b75f3ce9b827df28119319a4e71a2dc732908a network default NAD default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:27 [10.133.0.39/23] Jun 03 13:55:56.424092 ip-10-0-143-102 kubenswrapper[2573]: ' Jun 03 13:55:56.424092 ip-10-0-143-102 kubenswrapper[2573]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jun 03 13:55:56.424092 ip-10-0-143-102 kubenswrapper[2573]: > pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.431919 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.431860 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2hnr\" (UniqueName: \"kubernetes.io/projected/f2fed7a7-a594-4690-a0cf-b23c601c15d8-kube-api-access-w2hnr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-h2z7m\" (UID: \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:56.432031 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.431861 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw2vh\" (UniqueName: \"kubernetes.io/projected/c2a06da9-0d50-455d-a17e-fea1687e33df-kube-api-access-tw2vh\") pod \"kuadrant-operator-controller-manager-84b657d985-6v9vl\" (UID: \"c2a06da9-0d50-455d-a17e-fea1687e33df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:56.435460 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.435441 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:56.437972 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.437947 2573 status_manager.go:895] "Failed to get status for pod" podUID="373391b4-2053-41bd-a480-4f956764366a" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.487221 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.487147 2573 generic.go:358] "Generic (PLEG): container finished" podID="373391b4-2053-41bd-a480-4f956764366a" containerID="70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4" exitCode=0 Jun 03 13:55:56.487221 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.487195 2573 scope.go:117] "RemoveContainer" containerID="70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4" Jun 03 13:55:56.487383 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.487225 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:56.487383 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.487234 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.487383 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.487197 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" Jun 03 13:55:56.492012 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.491990 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:56.492801 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.492774 2573 status_manager.go:895] "Failed to get status for pod" podUID="373391b4-2053-41bd-a480-4f956764366a" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.495369 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.495352 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:56.495609 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.495596 2573 scope.go:117] "RemoveContainer" containerID="70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4" Jun 03 13:55:56.495838 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:55:56.495821 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4\": container with ID starting with 70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4 not found: ID does not exist" containerID="70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4" Jun 03 13:55:56.495916 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.495847 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4"} err="failed to get container status \"70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4\": rpc error: code = NotFound desc = could not find container \"70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4\": container with ID starting with 70771becf64e918c18a31d26fb329df2804c5c64083f23dced2d76774030d5c4 not found: ID does not exist" Jun 03 13:55:56.497123 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.497103 2573 status_manager.go:895] "Failed to get status for pod" podUID="ae720cc1-e83b-4793-8520-7917b0d17da4" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.500184 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.500162 2573 status_manager.go:895] "Failed to get status for pod" podUID="ae720cc1-e83b-4793-8520-7917b0d17da4" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.502759 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.502741 2573 status_manager.go:895] "Failed to get status for pod" podUID="8239e875-5f09-49ff-bd12-af3686f2927a" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" err="pods \"kuadrant-operator-controller-manager-84b657d985-mss6f\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.505012 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.504993 2573 status_manager.go:895] "Failed to get status for pod" podUID="373391b4-2053-41bd-a480-4f956764366a" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-q5vqg" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-q5vqg\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:56.521847 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.521830 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbn6l\" (UniqueName: \"kubernetes.io/projected/373391b4-2053-41bd-a480-4f956764366a-kube-api-access-zbn6l\") pod \"373391b4-2053-41bd-a480-4f956764366a\" (UID: \"373391b4-2053-41bd-a480-4f956764366a\") " Jun 03 13:55:56.521941 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.521921 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/373391b4-2053-41bd-a480-4f956764366a-extensions-socket-volume\") pod \"373391b4-2053-41bd-a480-4f956764366a\" (UID: \"373391b4-2053-41bd-a480-4f956764366a\") " Jun 03 13:55:56.522402 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.522383 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/373391b4-2053-41bd-a480-4f956764366a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "373391b4-2053-41bd-a480-4f956764366a" (UID: "373391b4-2053-41bd-a480-4f956764366a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:56.523842 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.523826 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/373391b4-2053-41bd-a480-4f956764366a-kube-api-access-zbn6l" (OuterVolumeSpecName: "kube-api-access-zbn6l") pod "373391b4-2053-41bd-a480-4f956764366a" (UID: "373391b4-2053-41bd-a480-4f956764366a"). InnerVolumeSpecName "kube-api-access-zbn6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:55:56.600874 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.600837 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:56.609648 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.609624 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:56.622635 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.622603 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae720cc1-e83b-4793-8520-7917b0d17da4-extensions-socket-volume\") pod \"ae720cc1-e83b-4793-8520-7917b0d17da4\" (UID: \"ae720cc1-e83b-4793-8520-7917b0d17da4\") " Jun 03 13:55:56.622761 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.622650 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8239e875-5f09-49ff-bd12-af3686f2927a-extensions-socket-volume\") pod \"8239e875-5f09-49ff-bd12-af3686f2927a\" (UID: \"8239e875-5f09-49ff-bd12-af3686f2927a\") " Jun 03 13:55:56.622761 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.622709 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nvfk\" (UniqueName: \"kubernetes.io/projected/8239e875-5f09-49ff-bd12-af3686f2927a-kube-api-access-8nvfk\") pod \"8239e875-5f09-49ff-bd12-af3686f2927a\" (UID: \"8239e875-5f09-49ff-bd12-af3686f2927a\") " Jun 03 13:55:56.622878 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.622775 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf69s\" (UniqueName: \"kubernetes.io/projected/ae720cc1-e83b-4793-8520-7917b0d17da4-kube-api-access-lf69s\") pod \"ae720cc1-e83b-4793-8520-7917b0d17da4\" (UID: \"ae720cc1-e83b-4793-8520-7917b0d17da4\") " Jun 03 13:55:56.622878 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.622813 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae720cc1-e83b-4793-8520-7917b0d17da4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ae720cc1-e83b-4793-8520-7917b0d17da4" (UID: "ae720cc1-e83b-4793-8520-7917b0d17da4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:56.622999 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.622949 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8239e875-5f09-49ff-bd12-af3686f2927a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8239e875-5f09-49ff-bd12-af3686f2927a" (UID: "8239e875-5f09-49ff-bd12-af3686f2927a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:55:56.623156 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.623129 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae720cc1-e83b-4793-8520-7917b0d17da4-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:56.623156 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.623153 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8239e875-5f09-49ff-bd12-af3686f2927a-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:56.623299 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.623164 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/373391b4-2053-41bd-a480-4f956764366a-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:56.623299 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.623173 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbn6l\" (UniqueName: \"kubernetes.io/projected/373391b4-2053-41bd-a480-4f956764366a-kube-api-access-zbn6l\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:56.624999 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.624975 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae720cc1-e83b-4793-8520-7917b0d17da4-kube-api-access-lf69s" (OuterVolumeSpecName: "kube-api-access-lf69s") pod "ae720cc1-e83b-4793-8520-7917b0d17da4" (UID: "ae720cc1-e83b-4793-8520-7917b0d17da4"). InnerVolumeSpecName "kube-api-access-lf69s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:55:56.625208 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.625173 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8239e875-5f09-49ff-bd12-af3686f2927a-kube-api-access-8nvfk" (OuterVolumeSpecName: "kube-api-access-8nvfk") pod "8239e875-5f09-49ff-bd12-af3686f2927a" (UID: "8239e875-5f09-49ff-bd12-af3686f2927a"). InnerVolumeSpecName "kube-api-access-8nvfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:55:56.724525 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.724496 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf69s\" (UniqueName: \"kubernetes.io/projected/ae720cc1-e83b-4793-8520-7917b0d17da4-kube-api-access-lf69s\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:56.724525 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.724525 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nvfk\" (UniqueName: \"kubernetes.io/projected/8239e875-5f09-49ff-bd12-af3686f2927a-kube-api-access-8nvfk\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:55:56.740232 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.740148 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="373391b4-2053-41bd-a480-4f956764366a" path="/var/lib/kubelet/pods/373391b4-2053-41bd-a480-4f956764366a/volumes" Jun 03 13:55:56.740715 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.740692 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8239e875-5f09-49ff-bd12-af3686f2927a" path="/var/lib/kubelet/pods/8239e875-5f09-49ff-bd12-af3686f2927a/volumes" Jun 03 13:55:56.741057 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.741039 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae720cc1-e83b-4793-8520-7917b0d17da4" path="/var/lib/kubelet/pods/ae720cc1-e83b-4793-8520-7917b0d17da4/volumes" Jun 03 13:55:56.747325 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.747293 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m"] Jun 03 13:55:56.748978 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:55:56.748954 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2fed7a7_a594_4690_a0cf_b23c601c15d8.slice/crio-67d4a586f4928e264a47c25f82fe0f54816c0cb2d1e7770890844486b46d82a5 WatchSource:0}: Error finding container 67d4a586f4928e264a47c25f82fe0f54816c0cb2d1e7770890844486b46d82a5: Status 404 returned error can't find the container with id 67d4a586f4928e264a47c25f82fe0f54816c0cb2d1e7770890844486b46d82a5 Jun 03 13:55:56.767366 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:56.767115 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl"] Jun 03 13:55:56.769193 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:55:56.769167 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a06da9_0d50_455d_a17e_fea1687e33df.slice/crio-be16221dcec7b202bf0819320339635e0a56e51cd94a60af723b4ab1621e6438 WatchSource:0}: Error finding container be16221dcec7b202bf0819320339635e0a56e51cd94a60af723b4ab1621e6438: Status 404 returned error can't find the container with id be16221dcec7b202bf0819320339635e0a56e51cd94a60af723b4ab1621e6438 Jun 03 13:55:57.493257 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.493224 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" event={"ID":"c2a06da9-0d50-455d-a17e-fea1687e33df","Type":"ContainerStarted","Data":"464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc"} Jun 03 13:55:57.493257 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.493260 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" event={"ID":"c2a06da9-0d50-455d-a17e-fea1687e33df","Type":"ContainerStarted","Data":"be16221dcec7b202bf0819320339635e0a56e51cd94a60af723b4ab1621e6438"} Jun 03 13:55:57.493507 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.493336 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:55:57.494765 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.494734 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" event={"ID":"f2fed7a7-a594-4690-a0cf-b23c601c15d8","Type":"ContainerStarted","Data":"d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638"} Jun 03 13:55:57.494921 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.494769 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" event={"ID":"f2fed7a7-a594-4690-a0cf-b23c601c15d8","Type":"ContainerStarted","Data":"67d4a586f4928e264a47c25f82fe0f54816c0cb2d1e7770890844486b46d82a5"} Jun 03 13:55:57.494921 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.494818 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:55:57.495709 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.495693 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" Jun 03 13:55:57.495799 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.495708 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" Jun 03 13:55:57.533782 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.533739 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" podStartSLOduration=1.5337264369999999 podStartE2EDuration="1.533726437s" podCreationTimestamp="2026-06-03 13:55:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:55:57.531311193 +0000 UTC m=+515.301999918" watchObservedRunningTime="2026-06-03 13:55:57.533726437 +0000 UTC m=+515.304415156" Jun 03 13:55:57.535427 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.535397 2573 status_manager.go:895] "Failed to get status for pod" podUID="ae720cc1-e83b-4793-8520-7917b0d17da4" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-ktbks" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-ktbks\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:57.537597 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.537572 2573 status_manager.go:895] "Failed to get status for pod" podUID="8239e875-5f09-49ff-bd12-af3686f2927a" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mss6f" err="pods \"kuadrant-operator-controller-manager-84b657d985-mss6f\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" Jun 03 13:55:57.585593 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:55:57.585541 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" podStartSLOduration=1.5855265429999998 podStartE2EDuration="1.585526543s" podCreationTimestamp="2026-06-03 13:55:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:55:57.580444547 +0000 UTC m=+515.351133271" watchObservedRunningTime="2026-06-03 13:55:57.585526543 +0000 UTC m=+515.356215268" Jun 03 13:56:08.500848 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.500774 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:56:08.500848 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.500824 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:56:08.676264 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.676227 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m"] Jun 03 13:56:08.676796 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.676761 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" podUID="f2fed7a7-a594-4690-a0cf-b23c601c15d8" containerName="manager" containerID="cri-o://d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638" gracePeriod=10 Jun 03 13:56:08.926423 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.926404 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:56:08.960145 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.960117 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55"] Jun 03 13:56:08.960424 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.960412 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2fed7a7-a594-4690-a0cf-b23c601c15d8" containerName="manager" Jun 03 13:56:08.960462 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.960425 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2fed7a7-a594-4690-a0cf-b23c601c15d8" containerName="manager" Jun 03 13:56:08.960495 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.960489 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="f2fed7a7-a594-4690-a0cf-b23c601c15d8" containerName="manager" Jun 03 13:56:08.962353 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.962338 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:08.989205 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:08.989177 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55"] Jun 03 13:56:09.025603 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.025564 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2fed7a7-a594-4690-a0cf-b23c601c15d8-extensions-socket-volume\") pod \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\" (UID: \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\") " Jun 03 13:56:09.025758 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.025614 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2hnr\" (UniqueName: \"kubernetes.io/projected/f2fed7a7-a594-4690-a0cf-b23c601c15d8-kube-api-access-w2hnr\") pod \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\" (UID: \"f2fed7a7-a594-4690-a0cf-b23c601c15d8\") " Jun 03 13:56:09.025850 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.025791 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk5hs\" (UniqueName: \"kubernetes.io/projected/5fac31af-1af3-424e-a040-0b169da59d05-kube-api-access-dk5hs\") pod \"kuadrant-operator-controller-manager-55c7f4c975-shx55\" (UID: \"5fac31af-1af3-424e-a040-0b169da59d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:09.026004 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.025978 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fed7a7-a594-4690-a0cf-b23c601c15d8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f2fed7a7-a594-4690-a0cf-b23c601c15d8" (UID: "f2fed7a7-a594-4690-a0cf-b23c601c15d8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:56:09.026004 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.025989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fac31af-1af3-424e-a040-0b169da59d05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-shx55\" (UID: \"5fac31af-1af3-424e-a040-0b169da59d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:09.026161 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.026045 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2fed7a7-a594-4690-a0cf-b23c601c15d8-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:56:09.028001 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.027976 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2fed7a7-a594-4690-a0cf-b23c601c15d8-kube-api-access-w2hnr" (OuterVolumeSpecName: "kube-api-access-w2hnr") pod "f2fed7a7-a594-4690-a0cf-b23c601c15d8" (UID: "f2fed7a7-a594-4690-a0cf-b23c601c15d8"). InnerVolumeSpecName "kube-api-access-w2hnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:56:09.127001 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.126919 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fac31af-1af3-424e-a040-0b169da59d05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-shx55\" (UID: \"5fac31af-1af3-424e-a040-0b169da59d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:09.127001 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.126963 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dk5hs\" (UniqueName: \"kubernetes.io/projected/5fac31af-1af3-424e-a040-0b169da59d05-kube-api-access-dk5hs\") pod \"kuadrant-operator-controller-manager-55c7f4c975-shx55\" (UID: \"5fac31af-1af3-424e-a040-0b169da59d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:09.127156 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.127023 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2hnr\" (UniqueName: \"kubernetes.io/projected/f2fed7a7-a594-4690-a0cf-b23c601c15d8-kube-api-access-w2hnr\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:56:09.127294 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.127271 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fac31af-1af3-424e-a040-0b169da59d05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-shx55\" (UID: \"5fac31af-1af3-424e-a040-0b169da59d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:09.141553 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.141529 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk5hs\" (UniqueName: \"kubernetes.io/projected/5fac31af-1af3-424e-a040-0b169da59d05-kube-api-access-dk5hs\") pod \"kuadrant-operator-controller-manager-55c7f4c975-shx55\" (UID: \"5fac31af-1af3-424e-a040-0b169da59d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:09.273407 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.273374 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:09.412749 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.412719 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55"] Jun 03 13:56:09.414637 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:56:09.414612 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fac31af_1af3_424e_a040_0b169da59d05.slice/crio-bd0194d10a96d6bc7bb5d640d179878f0a789ed75629e57ca7f1f9b18566c109 WatchSource:0}: Error finding container bd0194d10a96d6bc7bb5d640d179878f0a789ed75629e57ca7f1f9b18566c109: Status 404 returned error can't find the container with id bd0194d10a96d6bc7bb5d640d179878f0a789ed75629e57ca7f1f9b18566c109 Jun 03 13:56:09.538869 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.538833 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" event={"ID":"5fac31af-1af3-424e-a040-0b169da59d05","Type":"ContainerStarted","Data":"5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17"} Jun 03 13:56:09.538869 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.538876 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" event={"ID":"5fac31af-1af3-424e-a040-0b169da59d05","Type":"ContainerStarted","Data":"bd0194d10a96d6bc7bb5d640d179878f0a789ed75629e57ca7f1f9b18566c109"} Jun 03 13:56:09.539362 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.538918 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:09.540102 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.540073 2573 generic.go:358] "Generic (PLEG): container finished" podID="f2fed7a7-a594-4690-a0cf-b23c601c15d8" containerID="d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638" exitCode=0 Jun 03 13:56:09.540176 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.540123 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" Jun 03 13:56:09.540223 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.540119 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" event={"ID":"f2fed7a7-a594-4690-a0cf-b23c601c15d8","Type":"ContainerDied","Data":"d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638"} Jun 03 13:56:09.540268 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.540224 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m" event={"ID":"f2fed7a7-a594-4690-a0cf-b23c601c15d8","Type":"ContainerDied","Data":"67d4a586f4928e264a47c25f82fe0f54816c0cb2d1e7770890844486b46d82a5"} Jun 03 13:56:09.540268 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.540246 2573 scope.go:117] "RemoveContainer" containerID="d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638" Jun 03 13:56:09.549297 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.549278 2573 scope.go:117] "RemoveContainer" containerID="d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638" Jun 03 13:56:09.549639 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:56:09.549614 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638\": container with ID starting with d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638 not found: ID does not exist" containerID="d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638" Jun 03 13:56:09.549698 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.549643 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638"} err="failed to get container status \"d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638\": rpc error: code = NotFound desc = could not find container \"d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638\": container with ID starting with d734e240e8b5ba62a7a7ef3e9f7329bc7529cdfccee06e1eb2b03a2b5773c638 not found: ID does not exist" Jun 03 13:56:09.565781 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.565742 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" podStartSLOduration=1.565729902 podStartE2EDuration="1.565729902s" podCreationTimestamp="2026-06-03 13:56:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:56:09.562012113 +0000 UTC m=+527.332700838" watchObservedRunningTime="2026-06-03 13:56:09.565729902 +0000 UTC m=+527.336418627" Jun 03 13:56:09.584678 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.584645 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m"] Jun 03 13:56:09.603159 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:09.603132 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-h2z7m"] Jun 03 13:56:10.738488 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:10.738447 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2fed7a7-a594-4690-a0cf-b23c601c15d8" path="/var/lib/kubelet/pods/f2fed7a7-a594-4690-a0cf-b23c601c15d8/volumes" Jun 03 13:56:20.547567 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:20.547538 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 13:56:20.635079 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:20.635045 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl"] Jun 03 13:56:20.635395 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:20.635337 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" podUID="c2a06da9-0d50-455d-a17e-fea1687e33df" containerName="manager" containerID="cri-o://464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc" gracePeriod=10 Jun 03 13:56:20.879754 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:20.879732 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:56:21.024878 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.024840 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw2vh\" (UniqueName: \"kubernetes.io/projected/c2a06da9-0d50-455d-a17e-fea1687e33df-kube-api-access-tw2vh\") pod \"c2a06da9-0d50-455d-a17e-fea1687e33df\" (UID: \"c2a06da9-0d50-455d-a17e-fea1687e33df\") " Jun 03 13:56:21.025064 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.024947 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2a06da9-0d50-455d-a17e-fea1687e33df-extensions-socket-volume\") pod \"c2a06da9-0d50-455d-a17e-fea1687e33df\" (UID: \"c2a06da9-0d50-455d-a17e-fea1687e33df\") " Jun 03 13:56:21.025348 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.025325 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2a06da9-0d50-455d-a17e-fea1687e33df-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c2a06da9-0d50-455d-a17e-fea1687e33df" (UID: "c2a06da9-0d50-455d-a17e-fea1687e33df"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:56:21.026974 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.026947 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a06da9-0d50-455d-a17e-fea1687e33df-kube-api-access-tw2vh" (OuterVolumeSpecName: "kube-api-access-tw2vh") pod "c2a06da9-0d50-455d-a17e-fea1687e33df" (UID: "c2a06da9-0d50-455d-a17e-fea1687e33df"). InnerVolumeSpecName "kube-api-access-tw2vh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:56:21.125824 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.125725 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw2vh\" (UniqueName: \"kubernetes.io/projected/c2a06da9-0d50-455d-a17e-fea1687e33df-kube-api-access-tw2vh\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:56:21.125824 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.125770 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2a06da9-0d50-455d-a17e-fea1687e33df-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:56:21.594050 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.594015 2573 generic.go:358] "Generic (PLEG): container finished" podID="c2a06da9-0d50-455d-a17e-fea1687e33df" containerID="464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc" exitCode=0 Jun 03 13:56:21.594446 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.594073 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" Jun 03 13:56:21.594446 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.594103 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" event={"ID":"c2a06da9-0d50-455d-a17e-fea1687e33df","Type":"ContainerDied","Data":"464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc"} Jun 03 13:56:21.594446 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.594146 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl" event={"ID":"c2a06da9-0d50-455d-a17e-fea1687e33df","Type":"ContainerDied","Data":"be16221dcec7b202bf0819320339635e0a56e51cd94a60af723b4ab1621e6438"} Jun 03 13:56:21.594446 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.594166 2573 scope.go:117] "RemoveContainer" containerID="464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc" Jun 03 13:56:21.602483 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.602463 2573 scope.go:117] "RemoveContainer" containerID="464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc" Jun 03 13:56:21.602727 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:56:21.602710 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc\": container with ID starting with 464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc not found: ID does not exist" containerID="464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc" Jun 03 13:56:21.602787 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.602741 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc"} err="failed to get container status \"464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc\": rpc error: code = NotFound desc = could not find container \"464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc\": container with ID starting with 464f0e6e210e4db0ff62972cd01345b3ff3af7590509d07c3d565bc7193929cc not found: ID does not exist" Jun 03 13:56:21.622797 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.622763 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl"] Jun 03 13:56:21.627107 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:21.627085 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-6v9vl"] Jun 03 13:56:22.738686 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:22.738653 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2a06da9-0d50-455d-a17e-fea1687e33df" path="/var/lib/kubelet/pods/c2a06da9-0d50-455d-a17e-fea1687e33df/volumes" Jun 03 13:56:38.964076 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:38.964040 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-57vjf"] Jun 03 13:56:38.964490 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:38.964345 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a06da9-0d50-455d-a17e-fea1687e33df" containerName="manager" Jun 03 13:56:38.964490 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:38.964356 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a06da9-0d50-455d-a17e-fea1687e33df" containerName="manager" Jun 03 13:56:38.964490 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:38.964415 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="c2a06da9-0d50-455d-a17e-fea1687e33df" containerName="manager" Jun 03 13:56:38.967059 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:38.967044 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:38.969563 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:38.969535 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 03 13:56:38.969701 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:38.969637 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-57sf8\"" Jun 03 13:56:38.978040 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:38.978018 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-57vjf"] Jun 03 13:56:39.053509 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.053481 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-57vjf"] Jun 03 13:56:39.064792 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.064766 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t97v\" (UniqueName: \"kubernetes.io/projected/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-kube-api-access-5t97v\") pod \"limitador-limitador-7d549b5b-57vjf\" (UID: \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:39.064952 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.064824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-config-file\") pod \"limitador-limitador-7d549b5b-57vjf\" (UID: \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:39.165443 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.165402 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-config-file\") pod \"limitador-limitador-7d549b5b-57vjf\" (UID: \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:39.165610 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.165516 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5t97v\" (UniqueName: \"kubernetes.io/projected/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-kube-api-access-5t97v\") pod \"limitador-limitador-7d549b5b-57vjf\" (UID: \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:39.166139 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.166118 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-config-file\") pod \"limitador-limitador-7d549b5b-57vjf\" (UID: \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:39.174464 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.174433 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t97v\" (UniqueName: \"kubernetes.io/projected/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-kube-api-access-5t97v\") pod \"limitador-limitador-7d549b5b-57vjf\" (UID: \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:39.278353 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.278262 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:39.410681 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.410658 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-57vjf"] Jun 03 13:56:39.412964 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:56:39.412933 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89bcf4b7_5143_4ede_94ca_6e16fa74de1a.slice/crio-a5beebfce9425d7d13269ba3efaa4c706dd800fb44606feb554397a9833237ef WatchSource:0}: Error finding container a5beebfce9425d7d13269ba3efaa4c706dd800fb44606feb554397a9833237ef: Status 404 returned error can't find the container with id a5beebfce9425d7d13269ba3efaa4c706dd800fb44606feb554397a9833237ef Jun 03 13:56:39.661484 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:39.661442 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" event={"ID":"89bcf4b7-5143-4ede-94ca-6e16fa74de1a","Type":"ContainerStarted","Data":"a5beebfce9425d7d13269ba3efaa4c706dd800fb44606feb554397a9833237ef"} Jun 03 13:56:42.675409 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:42.675374 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" event={"ID":"89bcf4b7-5143-4ede-94ca-6e16fa74de1a","Type":"ContainerStarted","Data":"3ec3f28ca7793f8d3d5a7128061249c3fed2cea9012346cb3c3957b00a4c9716"} Jun 03 13:56:42.675773 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:42.675437 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:42.693476 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:42.693423 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" podStartSLOduration=1.781381278 podStartE2EDuration="4.693409123s" podCreationTimestamp="2026-06-03 13:56:38 +0000 UTC" firstStartedPulling="2026-06-03 13:56:39.414941178 +0000 UTC m=+557.185629884" lastFinishedPulling="2026-06-03 13:56:42.326969025 +0000 UTC m=+560.097657729" observedRunningTime="2026-06-03 13:56:42.690804247 +0000 UTC m=+560.461492985" watchObservedRunningTime="2026-06-03 13:56:42.693409123 +0000 UTC m=+560.464097848" Jun 03 13:56:53.680489 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:53.680457 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:56.020276 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:56.020238 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-57vjf"] Jun 03 13:56:56.020651 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:56.020437 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" podUID="89bcf4b7-5143-4ede-94ca-6e16fa74de1a" containerName="limitador" containerID="cri-o://3ec3f28ca7793f8d3d5a7128061249c3fed2cea9012346cb3c3957b00a4c9716" gracePeriod=30 Jun 03 13:56:56.731130 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:56.731094 2573 generic.go:358] "Generic (PLEG): container finished" podID="89bcf4b7-5143-4ede-94ca-6e16fa74de1a" containerID="3ec3f28ca7793f8d3d5a7128061249c3fed2cea9012346cb3c3957b00a4c9716" exitCode=0 Jun 03 13:56:56.731302 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:56.731169 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" event={"ID":"89bcf4b7-5143-4ede-94ca-6e16fa74de1a","Type":"ContainerDied","Data":"3ec3f28ca7793f8d3d5a7128061249c3fed2cea9012346cb3c3957b00a4c9716"} Jun 03 13:56:56.969553 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:56.969533 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:57.015663 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.015587 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-config-file\") pod \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\" (UID: \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\") " Jun 03 13:56:57.015793 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.015671 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t97v\" (UniqueName: \"kubernetes.io/projected/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-kube-api-access-5t97v\") pod \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\" (UID: \"89bcf4b7-5143-4ede-94ca-6e16fa74de1a\") " Jun 03 13:56:57.016029 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.016003 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-config-file" (OuterVolumeSpecName: "config-file") pod "89bcf4b7-5143-4ede-94ca-6e16fa74de1a" (UID: "89bcf4b7-5143-4ede-94ca-6e16fa74de1a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 13:56:57.017754 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.017736 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-kube-api-access-5t97v" (OuterVolumeSpecName: "kube-api-access-5t97v") pod "89bcf4b7-5143-4ede-94ca-6e16fa74de1a" (UID: "89bcf4b7-5143-4ede-94ca-6e16fa74de1a"). InnerVolumeSpecName "kube-api-access-5t97v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:56:57.116159 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.116124 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-config-file\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:56:57.116159 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.116158 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t97v\" (UniqueName: \"kubernetes.io/projected/89bcf4b7-5143-4ede-94ca-6e16fa74de1a-kube-api-access-5t97v\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:56:57.735502 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.735474 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" Jun 03 13:56:57.735691 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.735500 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-57vjf" event={"ID":"89bcf4b7-5143-4ede-94ca-6e16fa74de1a","Type":"ContainerDied","Data":"a5beebfce9425d7d13269ba3efaa4c706dd800fb44606feb554397a9833237ef"} Jun 03 13:56:57.735691 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.735539 2573 scope.go:117] "RemoveContainer" containerID="3ec3f28ca7793f8d3d5a7128061249c3fed2cea9012346cb3c3957b00a4c9716" Jun 03 13:56:57.757808 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.757781 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-57vjf"] Jun 03 13:56:57.765901 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:57.762378 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-57vjf"] Jun 03 13:56:58.738858 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:56:58.738824 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89bcf4b7-5143-4ede-94ca-6e16fa74de1a" path="/var/lib/kubelet/pods/89bcf4b7-5143-4ede-94ca-6e16fa74de1a/volumes" Jun 03 13:57:00.580253 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.580220 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-6bpvw"] Jun 03 13:57:00.580643 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.580532 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89bcf4b7-5143-4ede-94ca-6e16fa74de1a" containerName="limitador" Jun 03 13:57:00.580643 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.580543 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="89bcf4b7-5143-4ede-94ca-6e16fa74de1a" containerName="limitador" Jun 03 13:57:00.580643 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.580607 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="89bcf4b7-5143-4ede-94ca-6e16fa74de1a" containerName="limitador" Jun 03 13:57:00.585878 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.585850 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:00.588261 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.588235 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 03 13:57:00.588558 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.588540 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-6bpvw"] Jun 03 13:57:00.647446 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.647416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a3542deb-bec6-4d23-8771-3c6f3cb70f02-data\") pod \"postgres-68c4fbbd6f-6bpvw\" (UID: \"a3542deb-bec6-4d23-8771-3c6f3cb70f02\") " pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:00.647602 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.647495 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx9gs\" (UniqueName: \"kubernetes.io/projected/a3542deb-bec6-4d23-8771-3c6f3cb70f02-kube-api-access-kx9gs\") pod \"postgres-68c4fbbd6f-6bpvw\" (UID: \"a3542deb-bec6-4d23-8771-3c6f3cb70f02\") " pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:00.748486 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.748455 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kx9gs\" (UniqueName: \"kubernetes.io/projected/a3542deb-bec6-4d23-8771-3c6f3cb70f02-kube-api-access-kx9gs\") pod \"postgres-68c4fbbd6f-6bpvw\" (UID: \"a3542deb-bec6-4d23-8771-3c6f3cb70f02\") " pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:00.748638 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.748502 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a3542deb-bec6-4d23-8771-3c6f3cb70f02-data\") pod \"postgres-68c4fbbd6f-6bpvw\" (UID: \"a3542deb-bec6-4d23-8771-3c6f3cb70f02\") " pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:00.748836 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.748822 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a3542deb-bec6-4d23-8771-3c6f3cb70f02-data\") pod \"postgres-68c4fbbd6f-6bpvw\" (UID: \"a3542deb-bec6-4d23-8771-3c6f3cb70f02\") " pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:00.758436 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.758415 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx9gs\" (UniqueName: \"kubernetes.io/projected/a3542deb-bec6-4d23-8771-3c6f3cb70f02-kube-api-access-kx9gs\") pod \"postgres-68c4fbbd6f-6bpvw\" (UID: \"a3542deb-bec6-4d23-8771-3c6f3cb70f02\") " pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:00.899515 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:00.899490 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:01.022921 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:01.022868 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-6bpvw"] Jun 03 13:57:01.025230 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:57:01.025206 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3542deb_bec6_4d23_8771_3c6f3cb70f02.slice/crio-326cd0c17076ff7f670dd370b13d54741044b7eb4f9c607d10d1dc9502fc5b54 WatchSource:0}: Error finding container 326cd0c17076ff7f670dd370b13d54741044b7eb4f9c607d10d1dc9502fc5b54: Status 404 returned error can't find the container with id 326cd0c17076ff7f670dd370b13d54741044b7eb4f9c607d10d1dc9502fc5b54 Jun 03 13:57:01.752725 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:01.752689 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-6bpvw" event={"ID":"a3542deb-bec6-4d23-8771-3c6f3cb70f02","Type":"ContainerStarted","Data":"326cd0c17076ff7f670dd370b13d54741044b7eb4f9c607d10d1dc9502fc5b54"} Jun 03 13:57:03.671267 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:57:03.671229 2573 log.go:32] "ImageFsInfo from image service failed" err="rpc error: code = Unknown desc = get image fs info unable to get usage for /var/lib/containers/storage/overlay-images: get disk usage for path /var/lib/containers/storage/overlay-images: lstat /var/lib/containers/storage/overlay-images/.tmp-images.json3210913839: no such file or directory" Jun 03 13:57:03.671614 ip-10-0-143-102 kubenswrapper[2573]: E0603 13:57:03.671282 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get imageFs stats: missing image stats: nil" Jun 03 13:57:04.763527 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:04.763484 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-6bpvw" event={"ID":"a3542deb-bec6-4d23-8771-3c6f3cb70f02","Type":"ContainerStarted","Data":"5904ede9d3af330a8d601e8147fae222679a51d6a03b9c31f768b5fecd0b4b19"} Jun 03 13:57:04.764035 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:04.763652 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:04.783743 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:04.783682 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-6bpvw" podStartSLOduration=2.098948639 podStartE2EDuration="4.78366765s" podCreationTimestamp="2026-06-03 13:57:00 +0000 UTC" firstStartedPulling="2026-06-03 13:57:01.026420544 +0000 UTC m=+578.797109247" lastFinishedPulling="2026-06-03 13:57:03.711139544 +0000 UTC m=+581.481828258" observedRunningTime="2026-06-03 13:57:04.780763615 +0000 UTC m=+582.551452340" watchObservedRunningTime="2026-06-03 13:57:04.78366765 +0000 UTC m=+582.554356374" Jun 03 13:57:10.795545 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:10.795515 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-6bpvw" Jun 03 13:57:12.288929 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.288870 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k"] Jun 03 13:57:12.295418 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.295389 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.299096 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.299071 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vf5vf\"" Jun 03 13:57:12.299311 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.299106 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 13:57:12.299386 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.299192 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 13:57:12.301094 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.301073 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k"] Jun 03 13:57:12.446367 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.446329 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.446555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.446421 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzwcn\" (UniqueName: \"kubernetes.io/projected/ddfdfb24-0dfa-4827-a5e9-63952cadd462-kube-api-access-mzwcn\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.446555 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.446458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.547298 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.547224 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mzwcn\" (UniqueName: \"kubernetes.io/projected/ddfdfb24-0dfa-4827-a5e9-63952cadd462-kube-api-access-mzwcn\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.547298 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.547266 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.547470 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.547314 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.547671 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.547653 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.547708 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.547692 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.557001 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.556979 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzwcn\" (UniqueName: \"kubernetes.io/projected/ddfdfb24-0dfa-4827-a5e9-63952cadd462-kube-api-access-mzwcn\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.605183 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.605161 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:12.736599 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:57:12.736566 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfdfb24_0dfa_4827_a5e9_63952cadd462.slice/crio-53ef04f88f635ec6af110ce0e176ebac39ae830800911efe7874d9f8e11bc1b1 WatchSource:0}: Error finding container 53ef04f88f635ec6af110ce0e176ebac39ae830800911efe7874d9f8e11bc1b1: Status 404 returned error can't find the container with id 53ef04f88f635ec6af110ce0e176ebac39ae830800911efe7874d9f8e11bc1b1 Jun 03 13:57:12.739156 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.739133 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k"] Jun 03 13:57:12.791400 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:12.791373 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" event={"ID":"ddfdfb24-0dfa-4827-a5e9-63952cadd462","Type":"ContainerStarted","Data":"53ef04f88f635ec6af110ce0e176ebac39ae830800911efe7874d9f8e11bc1b1"} Jun 03 13:57:13.796373 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:13.796338 2573 generic.go:358] "Generic (PLEG): container finished" podID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerID="0edb6e52095b87afa1384822f40b56ea0b2170a8f21df75d9d767b81d5ec11c6" exitCode=0 Jun 03 13:57:13.796815 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:13.796425 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" event={"ID":"ddfdfb24-0dfa-4827-a5e9-63952cadd462","Type":"ContainerDied","Data":"0edb6e52095b87afa1384822f40b56ea0b2170a8f21df75d9d767b81d5ec11c6"} Jun 03 13:57:14.802097 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:14.802058 2573 generic.go:358] "Generic (PLEG): container finished" podID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerID="1deac4c9b09d3603859a68fb763c38c7f7504bf84aa87ee8ba95117a72a599ee" exitCode=0 Jun 03 13:57:14.802473 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:14.802124 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" event={"ID":"ddfdfb24-0dfa-4827-a5e9-63952cadd462","Type":"ContainerDied","Data":"1deac4c9b09d3603859a68fb763c38c7f7504bf84aa87ee8ba95117a72a599ee"} Jun 03 13:57:15.808115 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:15.808085 2573 generic.go:358] "Generic (PLEG): container finished" podID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerID="9913f7776e8fb694ab8be133e290ab6e990cdb27247d7cd3e4510a21f81a5630" exitCode=0 Jun 03 13:57:15.808487 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:15.808134 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" event={"ID":"ddfdfb24-0dfa-4827-a5e9-63952cadd462","Type":"ContainerDied","Data":"9913f7776e8fb694ab8be133e290ab6e990cdb27247d7cd3e4510a21f81a5630"} Jun 03 13:57:16.940770 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:16.940748 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:17.082746 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.082652 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzwcn\" (UniqueName: \"kubernetes.io/projected/ddfdfb24-0dfa-4827-a5e9-63952cadd462-kube-api-access-mzwcn\") pod \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " Jun 03 13:57:17.082922 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.082765 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-util\") pod \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " Jun 03 13:57:17.082922 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.082796 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-bundle\") pod \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\" (UID: \"ddfdfb24-0dfa-4827-a5e9-63952cadd462\") " Jun 03 13:57:17.083324 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.083297 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-bundle" (OuterVolumeSpecName: "bundle") pod "ddfdfb24-0dfa-4827-a5e9-63952cadd462" (UID: "ddfdfb24-0dfa-4827-a5e9-63952cadd462"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:57:17.084675 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.084653 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddfdfb24-0dfa-4827-a5e9-63952cadd462-kube-api-access-mzwcn" (OuterVolumeSpecName: "kube-api-access-mzwcn") pod "ddfdfb24-0dfa-4827-a5e9-63952cadd462" (UID: "ddfdfb24-0dfa-4827-a5e9-63952cadd462"). InnerVolumeSpecName "kube-api-access-mzwcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:57:17.088078 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.088036 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-util" (OuterVolumeSpecName: "util") pod "ddfdfb24-0dfa-4827-a5e9-63952cadd462" (UID: "ddfdfb24-0dfa-4827-a5e9-63952cadd462"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:57:17.184211 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.184172 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-util\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:57:17.184211 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.184205 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfdfb24-0dfa-4827-a5e9-63952cadd462-bundle\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:57:17.184211 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.184216 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzwcn\" (UniqueName: \"kubernetes.io/projected/ddfdfb24-0dfa-4827-a5e9-63952cadd462-kube-api-access-mzwcn\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 13:57:17.816357 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.816331 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" Jun 03 13:57:17.816517 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.816324 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a706vk5k" event={"ID":"ddfdfb24-0dfa-4827-a5e9-63952cadd462","Type":"ContainerDied","Data":"53ef04f88f635ec6af110ce0e176ebac39ae830800911efe7874d9f8e11bc1b1"} Jun 03 13:57:17.816517 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:17.816447 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53ef04f88f635ec6af110ce0e176ebac39ae830800911efe7874d9f8e11bc1b1" Jun 03 13:57:20.031275 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.031245 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-2mn2l"] Jun 03 13:57:20.031603 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.031560 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerName="pull" Jun 03 13:57:20.031603 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.031570 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerName="pull" Jun 03 13:57:20.031603 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.031584 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerName="extract" Jun 03 13:57:20.031603 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.031589 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerName="extract" Jun 03 13:57:20.031603 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.031599 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerName="util" Jun 03 13:57:20.031603 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.031604 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerName="util" Jun 03 13:57:20.031795 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.031655 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="ddfdfb24-0dfa-4827-a5e9-63952cadd462" containerName="extract" Jun 03 13:57:20.040683 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.040662 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-2mn2l" Jun 03 13:57:20.044840 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.044772 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-9czxx\"" Jun 03 13:57:20.044840 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.044819 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 03 13:57:20.045074 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.044819 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 03 13:57:20.046833 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.046784 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-2mn2l"] Jun 03 13:57:20.205144 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.205110 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x2pb\" (UniqueName: \"kubernetes.io/projected/e30a0522-cf10-4eca-bcab-5e416b22c972-kube-api-access-9x2pb\") pod \"keycloak-operator-7864d68cbd-2mn2l\" (UID: \"e30a0522-cf10-4eca-bcab-5e416b22c972\") " pod="keycloak-system/keycloak-operator-7864d68cbd-2mn2l" Jun 03 13:57:20.306193 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.306122 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9x2pb\" (UniqueName: \"kubernetes.io/projected/e30a0522-cf10-4eca-bcab-5e416b22c972-kube-api-access-9x2pb\") pod \"keycloak-operator-7864d68cbd-2mn2l\" (UID: \"e30a0522-cf10-4eca-bcab-5e416b22c972\") " pod="keycloak-system/keycloak-operator-7864d68cbd-2mn2l" Jun 03 13:57:20.326791 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.326761 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x2pb\" (UniqueName: \"kubernetes.io/projected/e30a0522-cf10-4eca-bcab-5e416b22c972-kube-api-access-9x2pb\") pod \"keycloak-operator-7864d68cbd-2mn2l\" (UID: \"e30a0522-cf10-4eca-bcab-5e416b22c972\") " pod="keycloak-system/keycloak-operator-7864d68cbd-2mn2l" Jun 03 13:57:20.351540 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.351523 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-2mn2l" Jun 03 13:57:20.496642 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.496615 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-2mn2l"] Jun 03 13:57:20.498747 ip-10-0-143-102 kubenswrapper[2573]: W0603 13:57:20.498716 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30a0522_cf10_4eca_bcab_5e416b22c972.slice/crio-2243efa6a57aacd822259035c0fc9376ea84f642500cdfd911a5422ab4cb894f WatchSource:0}: Error finding container 2243efa6a57aacd822259035c0fc9376ea84f642500cdfd911a5422ab4cb894f: Status 404 returned error can't find the container with id 2243efa6a57aacd822259035c0fc9376ea84f642500cdfd911a5422ab4cb894f Jun 03 13:57:20.828630 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:20.828601 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-2mn2l" event={"ID":"e30a0522-cf10-4eca-bcab-5e416b22c972","Type":"ContainerStarted","Data":"2243efa6a57aacd822259035c0fc9376ea84f642500cdfd911a5422ab4cb894f"} Jun 03 13:57:22.698528 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:22.698504 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 13:57:22.702789 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:22.702767 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 13:57:26.854848 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:26.854743 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-2mn2l" event={"ID":"e30a0522-cf10-4eca-bcab-5e416b22c972","Type":"ContainerStarted","Data":"2c3755aa8577960e521ae44cc2fef6d1aa12553de0dd8679db15bec8ea9b8260"} Jun 03 13:57:26.873900 ip-10-0-143-102 kubenswrapper[2573]: I0603 13:57:26.873839 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-7864d68cbd-2mn2l" podStartSLOduration=0.932276885 podStartE2EDuration="6.873826313s" podCreationTimestamp="2026-06-03 13:57:20 +0000 UTC" firstStartedPulling="2026-06-03 13:57:20.500065521 +0000 UTC m=+598.270754224" lastFinishedPulling="2026-06-03 13:57:26.441614945 +0000 UTC m=+604.212303652" observedRunningTime="2026-06-03 13:57:26.87151531 +0000 UTC m=+604.642204035" watchObservedRunningTime="2026-06-03 13:57:26.873826313 +0000 UTC m=+604.644515037" Jun 03 14:00:00.133011 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.132973 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29674920-r4bjn"] Jun 03 14:00:00.136453 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.136436 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" Jun 03 14:00:00.138665 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.138637 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-t2tmj\"" Jun 03 14:00:00.143095 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.143072 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29674920-r4bjn"] Jun 03 14:00:00.197050 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.197016 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d68fs\" (UniqueName: \"kubernetes.io/projected/02c420f4-9c37-4258-9d20-7065d836ddbe-kube-api-access-d68fs\") pod \"maas-api-key-cleanup-29674920-r4bjn\" (UID: \"02c420f4-9c37-4258-9d20-7065d836ddbe\") " pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" Jun 03 14:00:00.298542 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.298495 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d68fs\" (UniqueName: \"kubernetes.io/projected/02c420f4-9c37-4258-9d20-7065d836ddbe-kube-api-access-d68fs\") pod \"maas-api-key-cleanup-29674920-r4bjn\" (UID: \"02c420f4-9c37-4258-9d20-7065d836ddbe\") " pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" Jun 03 14:00:00.307135 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.307109 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d68fs\" (UniqueName: \"kubernetes.io/projected/02c420f4-9c37-4258-9d20-7065d836ddbe-kube-api-access-d68fs\") pod \"maas-api-key-cleanup-29674920-r4bjn\" (UID: \"02c420f4-9c37-4258-9d20-7065d836ddbe\") " pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" Jun 03 14:00:00.448581 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.448552 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" Jun 03 14:00:00.575457 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.575424 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29674920-r4bjn"] Jun 03 14:00:00.577062 ip-10-0-143-102 kubenswrapper[2573]: W0603 14:00:00.577030 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c420f4_9c37_4258_9d20_7065d836ddbe.slice/crio-b60e8ac10c7dbb5d131f5407ce4115b94e9454c1ca1563fca435889f35d8f3c7 WatchSource:0}: Error finding container b60e8ac10c7dbb5d131f5407ce4115b94e9454c1ca1563fca435889f35d8f3c7: Status 404 returned error can't find the container with id b60e8ac10c7dbb5d131f5407ce4115b94e9454c1ca1563fca435889f35d8f3c7 Jun 03 14:00:00.578788 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:00.578768 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 14:00:01.426160 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:01.426126 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" event={"ID":"02c420f4-9c37-4258-9d20-7065d836ddbe","Type":"ContainerStarted","Data":"b60e8ac10c7dbb5d131f5407ce4115b94e9454c1ca1563fca435889f35d8f3c7"} Jun 03 14:00:06.446961 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:06.446920 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" event={"ID":"02c420f4-9c37-4258-9d20-7065d836ddbe","Type":"ContainerStarted","Data":"8c83b97bb636281e892212091bd29fed74662a0dbd80e66e155623239cf72d4f"} Jun 03 14:00:06.464115 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:06.464064 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" podStartSLOduration=1.3152014269999999 podStartE2EDuration="6.464046382s" podCreationTimestamp="2026-06-03 14:00:00 +0000 UTC" firstStartedPulling="2026-06-03 14:00:00.578991763 +0000 UTC m=+758.349680476" lastFinishedPulling="2026-06-03 14:00:05.727836726 +0000 UTC m=+763.498525431" observedRunningTime="2026-06-03 14:00:06.462966795 +0000 UTC m=+764.233655520" watchObservedRunningTime="2026-06-03 14:00:06.464046382 +0000 UTC m=+764.234735142" Jun 03 14:00:07.451811 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:07.451770 2573 generic.go:358] "Generic (PLEG): container finished" podID="02c420f4-9c37-4258-9d20-7065d836ddbe" containerID="8c83b97bb636281e892212091bd29fed74662a0dbd80e66e155623239cf72d4f" exitCode=0 Jun 03 14:00:07.452291 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:07.451858 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" event={"ID":"02c420f4-9c37-4258-9d20-7065d836ddbe","Type":"ContainerDied","Data":"8c83b97bb636281e892212091bd29fed74662a0dbd80e66e155623239cf72d4f"} Jun 03 14:00:08.589481 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:08.589458 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" Jun 03 14:00:08.675752 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:08.675721 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d68fs\" (UniqueName: \"kubernetes.io/projected/02c420f4-9c37-4258-9d20-7065d836ddbe-kube-api-access-d68fs\") pod \"02c420f4-9c37-4258-9d20-7065d836ddbe\" (UID: \"02c420f4-9c37-4258-9d20-7065d836ddbe\") " Jun 03 14:00:08.677880 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:08.677856 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02c420f4-9c37-4258-9d20-7065d836ddbe-kube-api-access-d68fs" (OuterVolumeSpecName: "kube-api-access-d68fs") pod "02c420f4-9c37-4258-9d20-7065d836ddbe" (UID: "02c420f4-9c37-4258-9d20-7065d836ddbe"). InnerVolumeSpecName "kube-api-access-d68fs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 14:00:08.777246 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:08.777212 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d68fs\" (UniqueName: \"kubernetes.io/projected/02c420f4-9c37-4258-9d20-7065d836ddbe-kube-api-access-d68fs\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 14:00:09.460993 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:09.460965 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" Jun 03 14:00:09.461152 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:09.460966 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29674920-r4bjn" event={"ID":"02c420f4-9c37-4258-9d20-7065d836ddbe","Type":"ContainerDied","Data":"b60e8ac10c7dbb5d131f5407ce4115b94e9454c1ca1563fca435889f35d8f3c7"} Jun 03 14:00:09.461152 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:00:09.461072 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b60e8ac10c7dbb5d131f5407ce4115b94e9454c1ca1563fca435889f35d8f3c7" Jun 03 14:02:22.739696 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:02:22.739625 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:02:22.749175 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:02:22.749154 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:03:59.374299 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.374207 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74fb547c57-n5q2t"] Jun 03 14:03:59.376645 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.374944 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02c420f4-9c37-4258-9d20-7065d836ddbe" containerName="cleanup" Jun 03 14:03:59.376645 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.374963 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="02c420f4-9c37-4258-9d20-7065d836ddbe" containerName="cleanup" Jun 03 14:03:59.376645 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.375031 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="02c420f4-9c37-4258-9d20-7065d836ddbe" containerName="cleanup" Jun 03 14:03:59.377711 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.377694 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fb547c57-n5q2t" Jun 03 14:03:59.380140 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.380120 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 03 14:03:59.380245 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.380227 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z29ps\"" Jun 03 14:03:59.384382 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.384355 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74fb547c57-n5q2t"] Jun 03 14:03:59.452900 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.452859 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc8fx\" (UniqueName: \"kubernetes.io/projected/4e859d50-d5c8-4274-b75b-8265cb822e9a-kube-api-access-tc8fx\") pod \"maas-controller-74fb547c57-n5q2t\" (UID: \"4e859d50-d5c8-4274-b75b-8265cb822e9a\") " pod="opendatahub/maas-controller-74fb547c57-n5q2t" Jun 03 14:03:59.553750 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.553724 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tc8fx\" (UniqueName: \"kubernetes.io/projected/4e859d50-d5c8-4274-b75b-8265cb822e9a-kube-api-access-tc8fx\") pod \"maas-controller-74fb547c57-n5q2t\" (UID: \"4e859d50-d5c8-4274-b75b-8265cb822e9a\") " pod="opendatahub/maas-controller-74fb547c57-n5q2t" Jun 03 14:03:59.569205 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.569182 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc8fx\" (UniqueName: \"kubernetes.io/projected/4e859d50-d5c8-4274-b75b-8265cb822e9a-kube-api-access-tc8fx\") pod \"maas-controller-74fb547c57-n5q2t\" (UID: \"4e859d50-d5c8-4274-b75b-8265cb822e9a\") " pod="opendatahub/maas-controller-74fb547c57-n5q2t" Jun 03 14:03:59.690429 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.690408 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fb547c57-n5q2t" Jun 03 14:03:59.813598 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:03:59.813573 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74fb547c57-n5q2t"] Jun 03 14:03:59.815274 ip-10-0-143-102 kubenswrapper[2573]: W0603 14:03:59.815247 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e859d50_d5c8_4274_b75b_8265cb822e9a.slice/crio-6fa9de537bdd3870f86dcf6e2c16630e1a9e65216af9e41dc93a2940a2a2f583 WatchSource:0}: Error finding container 6fa9de537bdd3870f86dcf6e2c16630e1a9e65216af9e41dc93a2940a2a2f583: Status 404 returned error can't find the container with id 6fa9de537bdd3870f86dcf6e2c16630e1a9e65216af9e41dc93a2940a2a2f583 Jun 03 14:04:00.333601 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:04:00.333569 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fb547c57-n5q2t" event={"ID":"4e859d50-d5c8-4274-b75b-8265cb822e9a","Type":"ContainerStarted","Data":"6fa9de537bdd3870f86dcf6e2c16630e1a9e65216af9e41dc93a2940a2a2f583"} Jun 03 14:04:02.342461 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:04:02.342428 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fb547c57-n5q2t" event={"ID":"4e859d50-d5c8-4274-b75b-8265cb822e9a","Type":"ContainerStarted","Data":"656cbf4a3f9f1bcd05b5b023e01523cdd05bfde641c0bbe8f41e8ae37904159e"} Jun 03 14:04:02.342845 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:04:02.342604 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74fb547c57-n5q2t" Jun 03 14:04:02.360791 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:04:02.360748 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-74fb547c57-n5q2t" podStartSLOduration=0.919599552 podStartE2EDuration="3.360735366s" podCreationTimestamp="2026-06-03 14:03:59 +0000 UTC" firstStartedPulling="2026-06-03 14:03:59.816500683 +0000 UTC m=+997.587189386" lastFinishedPulling="2026-06-03 14:04:02.257636496 +0000 UTC m=+1000.028325200" observedRunningTime="2026-06-03 14:04:02.358479907 +0000 UTC m=+1000.129168632" watchObservedRunningTime="2026-06-03 14:04:02.360735366 +0000 UTC m=+1000.131424091" Jun 03 14:04:13.352665 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:04:13.352636 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74fb547c57-n5q2t" Jun 03 14:07:22.772877 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:07:22.772849 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:07:22.780477 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:07:22.780458 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:12:22.802933 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:12:22.802904 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:12:22.812431 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:12:22.812408 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:14:22.959527 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:22.959432 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55"] Jun 03 14:14:22.960128 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:22.959748 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" podUID="5fac31af-1af3-424e-a040-0b169da59d05" containerName="manager" containerID="cri-o://5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17" gracePeriod=10 Jun 03 14:14:23.219359 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.219299 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 14:14:23.221580 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.221560 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dk5hs\" (UniqueName: \"kubernetes.io/projected/5fac31af-1af3-424e-a040-0b169da59d05-kube-api-access-dk5hs\") pod \"5fac31af-1af3-424e-a040-0b169da59d05\" (UID: \"5fac31af-1af3-424e-a040-0b169da59d05\") " Jun 03 14:14:23.221651 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.221594 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fac31af-1af3-424e-a040-0b169da59d05-extensions-socket-volume\") pod \"5fac31af-1af3-424e-a040-0b169da59d05\" (UID: \"5fac31af-1af3-424e-a040-0b169da59d05\") " Jun 03 14:14:23.221951 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.221925 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fac31af-1af3-424e-a040-0b169da59d05-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5fac31af-1af3-424e-a040-0b169da59d05" (UID: "5fac31af-1af3-424e-a040-0b169da59d05"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 14:14:23.223656 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.223633 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fac31af-1af3-424e-a040-0b169da59d05-kube-api-access-dk5hs" (OuterVolumeSpecName: "kube-api-access-dk5hs") pod "5fac31af-1af3-424e-a040-0b169da59d05" (UID: "5fac31af-1af3-424e-a040-0b169da59d05"). InnerVolumeSpecName "kube-api-access-dk5hs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 14:14:23.322351 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.322315 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk5hs\" (UniqueName: \"kubernetes.io/projected/5fac31af-1af3-424e-a040-0b169da59d05-kube-api-access-dk5hs\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 14:14:23.322351 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.322347 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fac31af-1af3-424e-a040-0b169da59d05-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 14:14:23.714837 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.714798 2573 generic.go:358] "Generic (PLEG): container finished" podID="5fac31af-1af3-424e-a040-0b169da59d05" containerID="5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17" exitCode=0 Jun 03 14:14:23.715058 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.714879 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" Jun 03 14:14:23.715058 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.714895 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" event={"ID":"5fac31af-1af3-424e-a040-0b169da59d05","Type":"ContainerDied","Data":"5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17"} Jun 03 14:14:23.715058 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.714943 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55" event={"ID":"5fac31af-1af3-424e-a040-0b169da59d05","Type":"ContainerDied","Data":"bd0194d10a96d6bc7bb5d640d179878f0a789ed75629e57ca7f1f9b18566c109"} Jun 03 14:14:23.715058 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.714967 2573 scope.go:117] "RemoveContainer" containerID="5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17" Jun 03 14:14:23.724280 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.724260 2573 scope.go:117] "RemoveContainer" containerID="5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17" Jun 03 14:14:23.724521 ip-10-0-143-102 kubenswrapper[2573]: E0603 14:14:23.724504 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17\": container with ID starting with 5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17 not found: ID does not exist" containerID="5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17" Jun 03 14:14:23.724592 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.724528 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17"} err="failed to get container status \"5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17\": rpc error: code = NotFound desc = could not find container \"5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17\": container with ID starting with 5ed496697f7b51776044689825df1d366f8649b77dff1898ceb7448bd163de17 not found: ID does not exist" Jun 03 14:14:23.737419 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.737392 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55"] Jun 03 14:14:23.740941 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:23.740916 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-shx55"] Jun 03 14:14:24.738306 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:14:24.738273 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fac31af-1af3-424e-a040-0b169da59d05" path="/var/lib/kubelet/pods/5fac31af-1af3-424e-a040-0b169da59d05/volumes" Jun 03 14:15:00.139139 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.139098 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29674935-w7txz"] Jun 03 14:15:00.139571 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.139422 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fac31af-1af3-424e-a040-0b169da59d05" containerName="manager" Jun 03 14:15:00.139571 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.139432 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fac31af-1af3-424e-a040-0b169da59d05" containerName="manager" Jun 03 14:15:00.139571 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.139517 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="5fac31af-1af3-424e-a040-0b169da59d05" containerName="manager" Jun 03 14:15:00.141583 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.141567 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" Jun 03 14:15:00.144233 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.144217 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-t2tmj\"" Jun 03 14:15:00.149837 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.149817 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29674935-w7txz"] Jun 03 14:15:00.221656 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.221618 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j82v\" (UniqueName: \"kubernetes.io/projected/11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d-kube-api-access-5j82v\") pod \"maas-api-key-cleanup-29674935-w7txz\" (UID: \"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d\") " pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" Jun 03 14:15:00.322642 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.322611 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5j82v\" (UniqueName: \"kubernetes.io/projected/11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d-kube-api-access-5j82v\") pod \"maas-api-key-cleanup-29674935-w7txz\" (UID: \"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d\") " pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" Jun 03 14:15:00.338037 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.338012 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j82v\" (UniqueName: \"kubernetes.io/projected/11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d-kube-api-access-5j82v\") pod \"maas-api-key-cleanup-29674935-w7txz\" (UID: \"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d\") " pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" Jun 03 14:15:00.452970 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.452929 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" Jun 03 14:15:00.577777 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.577749 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29674935-w7txz"] Jun 03 14:15:00.579415 ip-10-0-143-102 kubenswrapper[2573]: W0603 14:15:00.579381 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11fd06a0_9a3d_45b7_9cce_c6bfc999eb9d.slice/crio-73654a444d6189998933e4fe04adbfbd3df60fd61882f32e185b08cba7641a63 WatchSource:0}: Error finding container 73654a444d6189998933e4fe04adbfbd3df60fd61882f32e185b08cba7641a63: Status 404 returned error can't find the container with id 73654a444d6189998933e4fe04adbfbd3df60fd61882f32e185b08cba7641a63 Jun 03 14:15:00.581198 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.581182 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 14:15:00.855294 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.855269 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" event={"ID":"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d","Type":"ContainerStarted","Data":"e8b3646b275977cf94ce96de04efd2b4e7ae870e1700b104b237757e9be8b96a"} Jun 03 14:15:00.855373 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.855301 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" event={"ID":"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d","Type":"ContainerStarted","Data":"73654a444d6189998933e4fe04adbfbd3df60fd61882f32e185b08cba7641a63"} Jun 03 14:15:00.871798 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:00.871760 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" podStartSLOduration=0.871746684 podStartE2EDuration="871.746684ms" podCreationTimestamp="2026-06-03 14:15:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 14:15:00.86848644 +0000 UTC m=+1658.639175165" watchObservedRunningTime="2026-06-03 14:15:00.871746684 +0000 UTC m=+1658.642435409" Jun 03 14:15:01.859724 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:01.859643 2573 generic.go:358] "Generic (PLEG): container finished" podID="11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d" containerID="e8b3646b275977cf94ce96de04efd2b4e7ae870e1700b104b237757e9be8b96a" exitCode=0 Jun 03 14:15:01.860106 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:01.859735 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" event={"ID":"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d","Type":"ContainerDied","Data":"e8b3646b275977cf94ce96de04efd2b4e7ae870e1700b104b237757e9be8b96a"} Jun 03 14:15:02.984983 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:02.984960 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" Jun 03 14:15:03.045792 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:03.045752 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j82v\" (UniqueName: \"kubernetes.io/projected/11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d-kube-api-access-5j82v\") pod \"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d\" (UID: \"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d\") " Jun 03 14:15:03.047731 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:03.047699 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d-kube-api-access-5j82v" (OuterVolumeSpecName: "kube-api-access-5j82v") pod "11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d" (UID: "11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d"). InnerVolumeSpecName "kube-api-access-5j82v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 14:15:03.146989 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:03.146961 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j82v\" (UniqueName: \"kubernetes.io/projected/11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d-kube-api-access-5j82v\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" Jun 03 14:15:03.868927 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:03.868861 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" event={"ID":"11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d","Type":"ContainerDied","Data":"73654a444d6189998933e4fe04adbfbd3df60fd61882f32e185b08cba7641a63"} Jun 03 14:15:03.868927 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:03.868926 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73654a444d6189998933e4fe04adbfbd3df60fd61882f32e185b08cba7641a63" Jun 03 14:15:03.869169 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:03.868907 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29674935-w7txz" Jun 03 14:15:29.087830 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.087795 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst"] Jun 03 14:15:29.088250 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.088195 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d" containerName="cleanup" Jun 03 14:15:29.088250 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.088208 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d" containerName="cleanup" Jun 03 14:15:29.088325 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.088272 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d" containerName="cleanup" Jun 03 14:15:29.090614 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.090596 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.093300 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.093279 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jcfwk\"" Jun 03 14:15:29.093410 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.093283 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 14:15:29.094116 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.094096 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 14:15:29.108825 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.108802 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst"] Jun 03 14:15:29.141791 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.141758 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76451b21-7d48-4961-accd-1f2ec8a1dd94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hbmst\" (UID: \"76451b21-7d48-4961-accd-1f2ec8a1dd94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.141957 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.141869 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8jpf\" (UniqueName: \"kubernetes.io/projected/76451b21-7d48-4961-accd-1f2ec8a1dd94-kube-api-access-k8jpf\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hbmst\" (UID: \"76451b21-7d48-4961-accd-1f2ec8a1dd94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.242733 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.242702 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k8jpf\" (UniqueName: \"kubernetes.io/projected/76451b21-7d48-4961-accd-1f2ec8a1dd94-kube-api-access-k8jpf\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hbmst\" (UID: \"76451b21-7d48-4961-accd-1f2ec8a1dd94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.242911 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.242757 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76451b21-7d48-4961-accd-1f2ec8a1dd94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hbmst\" (UID: \"76451b21-7d48-4961-accd-1f2ec8a1dd94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.243161 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.243139 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76451b21-7d48-4961-accd-1f2ec8a1dd94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hbmst\" (UID: \"76451b21-7d48-4961-accd-1f2ec8a1dd94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.252192 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.252162 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8jpf\" (UniqueName: \"kubernetes.io/projected/76451b21-7d48-4961-accd-1f2ec8a1dd94-kube-api-access-k8jpf\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hbmst\" (UID: \"76451b21-7d48-4961-accd-1f2ec8a1dd94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.401087 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.401063 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.532173 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.532139 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst"] Jun 03 14:15:29.533690 ip-10-0-143-102 kubenswrapper[2573]: W0603 14:15:29.533659 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76451b21_7d48_4961_accd_1f2ec8a1dd94.slice/crio-2f54f44809c74368ac661cd60990fcfd6ece806969d9ba8490efb2bc3c6c7b1a WatchSource:0}: Error finding container 2f54f44809c74368ac661cd60990fcfd6ece806969d9ba8490efb2bc3c6c7b1a: Status 404 returned error can't find the container with id 2f54f44809c74368ac661cd60990fcfd6ece806969d9ba8490efb2bc3c6c7b1a Jun 03 14:15:29.970109 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.970065 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" event={"ID":"76451b21-7d48-4961-accd-1f2ec8a1dd94","Type":"ContainerStarted","Data":"81c6c1b125581b74f1c7afaeb230347d2a50054993dae09f72565bd91e56d478"} Jun 03 14:15:29.970109 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.970114 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" event={"ID":"76451b21-7d48-4961-accd-1f2ec8a1dd94","Type":"ContainerStarted","Data":"2f54f44809c74368ac661cd60990fcfd6ece806969d9ba8490efb2bc3c6c7b1a"} Jun 03 14:15:29.970327 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.970135 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:15:29.990587 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:29.990532 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" podStartSLOduration=0.990518914 podStartE2EDuration="990.518914ms" podCreationTimestamp="2026-06-03 14:15:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 14:15:29.98911575 +0000 UTC m=+1687.759804475" watchObservedRunningTime="2026-06-03 14:15:29.990518914 +0000 UTC m=+1687.761207640" Jun 03 14:15:40.976584 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:15:40.976552 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hbmst" Jun 03 14:17:22.833925 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:17:22.833874 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:17:22.845146 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:17:22.845121 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:22:22.864671 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:22:22.864640 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:22:22.875223 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:22:22.875204 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:25:35.365915 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:35.365870 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-d48bdd865-m5p2x_a3a85c26-b143-49d0-b78a-d02bf1c8bec6/nginx/0.log" Jun 03 14:25:35.606903 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:35.606855 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-qcbwg_c283a5e9-49ce-4120-84a9-0eb25761276a/manager/0.log" Jun 03 14:25:35.729852 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:35.729827 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-522nb_beaec3fb-4774-4051-9de2-cdc52b5f9a73/manager/0.log" Jun 03 14:25:35.953510 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:35.953483 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29674920-r4bjn_02c420f4-9c37-4258-9d20-7065d836ddbe/cleanup/0.log" Jun 03 14:25:36.062161 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:36.062090 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29674935-w7txz_11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d/cleanup/0.log" Jun 03 14:25:36.182638 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:36.182611 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74fb547c57-n5q2t_4e859d50-d5c8-4274-b75b-8265cb822e9a/manager/0.log" Jun 03 14:25:36.292231 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:36.292210 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-2q2ft_d0652c0c-1462-446a-a77a-add5ae61b3a0/server/0.log" Jun 03 14:25:36.412633 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:36.412606 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-bb9mp_1f638895-30e1-4010-91d1-5f379b498f3b/manager/0.log" Jun 03 14:25:36.648028 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:36.648004 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5496d4666c-dlkbf_ebc44fb7-a947-4b67-8f6d-1f17a9577814/copy-manifests/0.log" Jun 03 14:25:36.655368 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:36.655345 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5496d4666c-dlkbf_ebc44fb7-a947-4b67-8f6d-1f17a9577814/manager/0.log" Jun 03 14:25:36.889879 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:36.889801 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-6bpvw_a3542deb-bec6-4d23-8771-3c6f3cb70f02/postgres/0.log" Jun 03 14:25:37.657007 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.656979 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf_04b928fb-3c40-4bcd-907d-743420427f1c/util/0.log" Jun 03 14:25:37.663294 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.663273 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf_04b928fb-3c40-4bcd-907d-743420427f1c/pull/0.log" Jun 03 14:25:37.669050 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.669026 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf_04b928fb-3c40-4bcd-907d-743420427f1c/extract/0.log" Jun 03 14:25:37.780331 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.780311 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb_96f6f681-a4c0-49cc-9020-98d39183580d/util/0.log" Jun 03 14:25:37.787522 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.787504 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb_96f6f681-a4c0-49cc-9020-98d39183580d/pull/0.log" Jun 03 14:25:37.796325 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.796303 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb_96f6f681-a4c0-49cc-9020-98d39183580d/extract/0.log" Jun 03 14:25:37.910760 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.910702 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz_a993f6dd-bcb4-4125-8be9-6504e72151c2/util/0.log" Jun 03 14:25:37.918225 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.918200 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz_a993f6dd-bcb4-4125-8be9-6504e72151c2/pull/0.log" Jun 03 14:25:37.925068 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:37.925050 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz_a993f6dd-bcb4-4125-8be9-6504e72151c2/extract/0.log" Jun 03 14:25:38.048115 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:38.048092 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z_af0351b6-5c0c-413d-ba43-970f487699be/util/0.log" Jun 03 14:25:38.055334 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:38.055298 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z_af0351b6-5c0c-413d-ba43-970f487699be/pull/0.log" Jun 03 14:25:38.062400 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:38.062379 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z_af0351b6-5c0c-413d-ba43-970f487699be/extract/0.log" Jun 03 14:25:38.790760 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:38.790727 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-hbmst_76451b21-7d48-4961-accd-1f2ec8a1dd94/manager/0.log" Jun 03 14:25:39.498425 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:39.498393 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-wbsnk_e06ac003-0404-4470-915d-b9d0dc13ed92/discovery/0.log" Jun 03 14:25:39.720456 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:39.720426 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-854c9b8647-m9j5g_ac1e6c67-5211-4357-92ca-261541bcb148/kube-auth-proxy/0.log" Jun 03 14:25:48.714350 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:48.714313 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j4b9r_a7566a4f-1ff5-45ae-93fe-7c91e3223f28/global-pull-secret-syncer/0.log" Jun 03 14:25:48.812331 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:48.812302 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m8ltt_9120bfb5-d3f1-4385-aa41-d563cc5697a0/konnectivity-agent/0.log" Jun 03 14:25:48.957899 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:48.957852 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-102.ec2.internal_ab071b712deb3518b2ecc312d28f3334/haproxy/0.log" Jun 03 14:25:52.807065 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:52.807035 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf_04b928fb-3c40-4bcd-907d-743420427f1c/extract/0.log" Jun 03 14:25:52.832960 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:52.832934 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf_04b928fb-3c40-4bcd-907d-743420427f1c/util/0.log" Jun 03 14:25:52.865112 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:52.865083 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75986ztf_04b928fb-3c40-4bcd-907d-743420427f1c/pull/0.log" Jun 03 14:25:52.890834 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:52.890818 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb_96f6f681-a4c0-49cc-9020-98d39183580d/extract/0.log" Jun 03 14:25:52.913178 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:52.913160 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb_96f6f681-a4c0-49cc-9020-98d39183580d/util/0.log" Jun 03 14:25:52.937514 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:52.937496 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lx8jb_96f6f681-a4c0-49cc-9020-98d39183580d/pull/0.log" Jun 03 14:25:52.976547 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:52.976525 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz_a993f6dd-bcb4-4125-8be9-6504e72151c2/extract/0.log" Jun 03 14:25:53.017725 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:53.017706 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz_a993f6dd-bcb4-4125-8be9-6504e72151c2/util/0.log" Jun 03 14:25:53.041866 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:53.041842 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rcmfz_a993f6dd-bcb4-4125-8be9-6504e72151c2/pull/0.log" Jun 03 14:25:53.071405 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:53.071332 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z_af0351b6-5c0c-413d-ba43-970f487699be/extract/0.log" Jun 03 14:25:53.093901 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:53.093858 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z_af0351b6-5c0c-413d-ba43-970f487699be/util/0.log" Jun 03 14:25:53.116543 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:53.116523 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5q6z_af0351b6-5c0c-413d-ba43-970f487699be/pull/0.log" Jun 03 14:25:53.674863 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:53.674826 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-hbmst_76451b21-7d48-4961-accd-1f2ec8a1dd94/manager/0.log" Jun 03 14:25:55.708198 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:55.708121 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgxs9_267ee72b-2613-4f92-bbb0-8af92e3ca4f2/node-exporter/0.log" Jun 03 14:25:55.730673 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:55.730651 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgxs9_267ee72b-2613-4f92-bbb0-8af92e3ca4f2/kube-rbac-proxy/0.log" Jun 03 14:25:55.754673 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:55.754655 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgxs9_267ee72b-2613-4f92-bbb0-8af92e3ca4f2/init-textfile/0.log" Jun 03 14:25:58.593706 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:58.593674 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-vtbnp_4bef7fe1-c7ed-429f-bb84-80b0871b530c/download-server/0.log" Jun 03 14:25:59.800110 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.800074 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s"] Jun 03 14:25:59.803893 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.803850 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.806495 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.806470 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wb4xx\"/\"kube-root-ca.crt\"" Jun 03 14:25:59.806633 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.806532 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wb4xx\"/\"openshift-service-ca.crt\"" Jun 03 14:25:59.806633 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.806544 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wb4xx\"/\"default-dockercfg-nhtth\"" Jun 03 14:25:59.813746 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.813724 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s"] Jun 03 14:25:59.896610 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.896584 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-podres\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.896722 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.896637 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlmm4\" (UniqueName: \"kubernetes.io/projected/5c4488e4-9e56-4c6a-9068-d536260f70c3-kube-api-access-tlmm4\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.896722 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.896688 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-proc\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.896796 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.896719 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-lib-modules\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.896796 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.896744 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-sys\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997279 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997252 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-podres\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997411 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997294 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tlmm4\" (UniqueName: \"kubernetes.io/projected/5c4488e4-9e56-4c6a-9068-d536260f70c3-kube-api-access-tlmm4\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997411 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997323 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-proc\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997411 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997350 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-lib-modules\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997411 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997372 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-sys\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997617 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997435 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-podres\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997617 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997470 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-sys\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997617 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997490 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-lib-modules\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:25:59.997617 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:25:59.997568 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c4488e4-9e56-4c6a-9068-d536260f70c3-proc\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:26:00.005772 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.005744 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlmm4\" (UniqueName: \"kubernetes.io/projected/5c4488e4-9e56-4c6a-9068-d536260f70c3-kube-api-access-tlmm4\") pod \"perf-node-gather-daemonset-wr95s\" (UID: \"5c4488e4-9e56-4c6a-9068-d536260f70c3\") " pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:26:00.094977 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.094879 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cmq8g_7ba04155-ffdc-4d87-be26-adc7e6e7bbd6/dns/0.log" Jun 03 14:26:00.116351 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.116325 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:26:00.119728 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.119710 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cmq8g_7ba04155-ffdc-4d87-be26-adc7e6e7bbd6/kube-rbac-proxy/0.log" Jun 03 14:26:00.223612 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.223570 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dqxqv_8c8e8e92-11f2-4735-a6bc-32c4bc13f517/dns-node-resolver/0.log" Jun 03 14:26:00.254427 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.254390 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s"] Jun 03 14:26:00.255771 ip-10-0-143-102 kubenswrapper[2573]: W0603 14:26:00.255750 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5c4488e4_9e56_4c6a_9068_d536260f70c3.slice/crio-b81d27ab579ede01e5dd535d461a3982a288760141a85f2f052258a07c6a72a3 WatchSource:0}: Error finding container b81d27ab579ede01e5dd535d461a3982a288760141a85f2f052258a07c6a72a3: Status 404 returned error can't find the container with id b81d27ab579ede01e5dd535d461a3982a288760141a85f2f052258a07c6a72a3 Jun 03 14:26:00.257687 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.257671 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 14:26:00.359366 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.359338 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" event={"ID":"5c4488e4-9e56-4c6a-9068-d536260f70c3","Type":"ContainerStarted","Data":"075d8162ef875a8e97929a9d94fba5350ca1eee4e11bb091f74a7a2dec8b1f26"} Jun 03 14:26:00.359453 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.359368 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" event={"ID":"5c4488e4-9e56-4c6a-9068-d536260f70c3","Type":"ContainerStarted","Data":"b81d27ab579ede01e5dd535d461a3982a288760141a85f2f052258a07c6a72a3"} Jun 03 14:26:00.359453 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.359392 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:26:00.393404 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.393364 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" podStartSLOduration=1.393353161 podStartE2EDuration="1.393353161s" podCreationTimestamp="2026-06-03 14:25:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 14:26:00.391100775 +0000 UTC m=+2318.161789499" watchObservedRunningTime="2026-06-03 14:26:00.393353161 +0000 UTC m=+2318.164041886" Jun 03 14:26:00.896625 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:00.896592 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9ghdz_c78e1418-cf8d-4609-a4a8-f12b567f096d/node-ca/0.log" Jun 03 14:26:01.956265 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:01.956232 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-wbsnk_e06ac003-0404-4470-915d-b9d0dc13ed92/discovery/0.log" Jun 03 14:26:02.005835 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:02.005808 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-854c9b8647-m9j5g_ac1e6c67-5211-4357-92ca-261541bcb148/kube-auth-proxy/0.log" Jun 03 14:26:02.713734 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:02.713689 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7ptv9_72f705bc-93cc-4b36-bd92-ef3ad046561e/serve-healthcheck-canary/0.log" Jun 03 14:26:03.305194 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:03.305167 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dn9ds_3042a477-badc-4cb2-9b86-ae27a9649d57/kube-rbac-proxy/0.log" Jun 03 14:26:03.329259 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:03.329233 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dn9ds_3042a477-badc-4cb2-9b86-ae27a9649d57/exporter/0.log" Jun 03 14:26:03.352528 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:03.352498 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dn9ds_3042a477-badc-4cb2-9b86-ae27a9649d57/extractor/0.log" Jun 03 14:26:05.579588 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:05.579557 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-d48bdd865-m5p2x_a3a85c26-b143-49d0-b78a-d02bf1c8bec6/nginx/0.log" Jun 03 14:26:05.628314 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:05.628288 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-qcbwg_c283a5e9-49ce-4120-84a9-0eb25761276a/manager/0.log" Jun 03 14:26:05.754713 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:05.754685 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-522nb_beaec3fb-4774-4051-9de2-cdc52b5f9a73/manager/0.log" Jun 03 14:26:05.810713 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:05.810687 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29674920-r4bjn_02c420f4-9c37-4258-9d20-7065d836ddbe/cleanup/0.log" Jun 03 14:26:05.836391 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:05.836333 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29674935-w7txz_11fd06a0-9a3d-45b7-9cce-c6bfc999eb9d/cleanup/0.log" Jun 03 14:26:05.947955 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:05.947925 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74fb547c57-n5q2t_4e859d50-d5c8-4274-b75b-8265cb822e9a/manager/0.log" Jun 03 14:26:05.970054 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:05.970027 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-2q2ft_d0652c0c-1462-446a-a77a-add5ae61b3a0/server/0.log" Jun 03 14:26:06.043767 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:06.043740 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-bb9mp_1f638895-30e1-4010-91d1-5f379b498f3b/manager/0.log" Jun 03 14:26:06.130222 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:06.130153 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5496d4666c-dlkbf_ebc44fb7-a947-4b67-8f6d-1f17a9577814/manager/0.log" Jun 03 14:26:06.153308 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:06.153288 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5496d4666c-dlkbf_ebc44fb7-a947-4b67-8f6d-1f17a9577814/copy-manifests/0.log" Jun 03 14:26:06.331921 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:06.331897 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-6bpvw_a3542deb-bec6-4d23-8771-3c6f3cb70f02/postgres/0.log" Jun 03 14:26:06.372437 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:06.372415 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wb4xx/perf-node-gather-daemonset-wr95s" Jun 03 14:26:12.827400 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:12.827371 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-lrfnl_5af435e1-70e4-4200-86c4-3fccc88e9f0f/migrator/0.log" Jun 03 14:26:12.851942 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:12.851914 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-lrfnl_5af435e1-70e4-4200-86c4-3fccc88e9f0f/graceful-termination/0.log" Jun 03 14:26:14.726768 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:14.726731 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s44x2_ee004359-f795-4e63-85b4-9a4a3f0765ed/kube-multus-additional-cni-plugins/0.log" Jun 03 14:26:14.758334 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:14.758304 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s44x2_ee004359-f795-4e63-85b4-9a4a3f0765ed/egress-router-binary-copy/0.log" Jun 03 14:26:14.781732 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:14.781712 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s44x2_ee004359-f795-4e63-85b4-9a4a3f0765ed/cni-plugins/0.log" Jun 03 14:26:14.804168 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:14.804144 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s44x2_ee004359-f795-4e63-85b4-9a4a3f0765ed/bond-cni-plugin/0.log" Jun 03 14:26:14.835446 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:14.835428 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s44x2_ee004359-f795-4e63-85b4-9a4a3f0765ed/routeoverride-cni/0.log" Jun 03 14:26:14.861743 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:14.861724 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s44x2_ee004359-f795-4e63-85b4-9a4a3f0765ed/whereabouts-cni-bincopy/0.log" Jun 03 14:26:14.886308 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:14.886291 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s44x2_ee004359-f795-4e63-85b4-9a4a3f0765ed/whereabouts-cni/0.log" Jun 03 14:26:14.956848 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:14.956818 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-x7rlv_42ca3328-c9ec-44cc-8ec5-966ce538afce/kube-multus/0.log" Jun 03 14:26:15.040925 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:15.040830 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9nbjd_2c98fc98-abf1-4502-9f2d-cd8bcd1968db/network-metrics-daemon/0.log" Jun 03 14:26:15.072371 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:15.072351 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9nbjd_2c98fc98-abf1-4502-9f2d-cd8bcd1968db/kube-rbac-proxy/0.log" Jun 03 14:26:16.352390 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.352359 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-controller/0.log" Jun 03 14:26:16.374477 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.374448 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/0.log" Jun 03 14:26:16.394507 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.394469 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovn-acl-logging/1.log" Jun 03 14:26:16.415380 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.415359 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/kube-rbac-proxy-node/0.log" Jun 03 14:26:16.444724 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.444707 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/kube-rbac-proxy-ovn-metrics/0.log" Jun 03 14:26:16.471288 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.471270 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/northd/0.log" Jun 03 14:26:16.496767 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.496749 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/nbdb/0.log" Jun 03 14:26:16.526357 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.526334 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/sbdb/0.log" Jun 03 14:26:16.710082 ip-10-0-143-102 kubenswrapper[2573]: I0603 14:26:16.710052 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bznnp_31fb2514-a18b-41df-b762-2c97c0da1b6c/ovnkube-controller/0.log"