Jun 09 02:56:39.966259 ip-10-0-130-143 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 09 02:56:39.966390 ip-10-0-130-143 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 09 02:56:39.966473 ip-10-0-130-143 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 09 02:56:39.966890 ip-10-0-130-143 systemd[1]: Failed to start Kubernetes Kubelet. Jun 09 02:56:49.995781 ip-10-0-130-143 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 09 02:56:49.995795 ip-10-0-130-143 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 639054eefcc54a9197effa51ccf4b75b -- Jun 09 02:59:25.983504 ip-10-0-130-143 systemd[1]: Starting Kubernetes Kubelet... Jun 09 02:59:26.389847 ip-10-0-130-143 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 02:59:26.389847 ip-10-0-130-143 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 09 02:59:26.389847 ip-10-0-130-143 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 02:59:26.389847 ip-10-0-130-143 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 09 02:59:26.389847 ip-10-0-130-143 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 02:59:26.391273 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.391171 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 09 02:59:26.394740 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394725 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 02:59:26.394740 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394740 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394744 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394747 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394750 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394752 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394755 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394762 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394766 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394768 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394771 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394774 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394777 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394780 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394783 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394785 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394788 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394791 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394794 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394796 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394799 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394801 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 09 02:59:26.394813 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394804 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394807 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394810 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394812 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394815 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394817 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394820 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394823 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394826 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394829 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394832 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394834 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394837 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394839 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394842 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394845 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394848 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394852 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394855 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 02:59:26.395370 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394858 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394862 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394864 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394867 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394871 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394874 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394877 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394880 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394883 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394885 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394888 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394891 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394893 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394895 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394898 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394902 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394904 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394907 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394910 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 02:59:26.395849 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394913 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394915 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394919 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394922 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394924 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394927 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394929 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394932 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394936 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394939 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394942 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394945 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394948 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394951 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394954 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394956 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394959 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394961 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394964 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 02:59:26.396313 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394966 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394970 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394972 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394975 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394978 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394980 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394983 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394985 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394988 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394991 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394994 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.394997 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395000 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395002 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395004 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395007 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395009 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395012 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395014 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395016 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 02:59:26.396765 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395800 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395808 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395813 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395816 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395819 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395823 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395826 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395829 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395831 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395834 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395837 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395840 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395843 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395846 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395848 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395851 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395854 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395857 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395860 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395863 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 02:59:26.397251 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395866 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395869 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395872 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395875 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395877 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395880 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395882 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395885 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395888 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395891 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395894 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395896 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395899 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395902 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395904 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395907 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395910 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395912 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395916 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395918 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 02:59:26.397775 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395921 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395923 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395925 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395928 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395931 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395933 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395936 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395938 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395942 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395944 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395947 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395949 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395952 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395955 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395957 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395960 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395963 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395966 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395968 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395971 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 02:59:26.398274 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395973 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395976 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395979 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395983 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395987 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395989 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395992 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395995 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.395998 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396000 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396003 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396007 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396010 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396013 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396016 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396019 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396021 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396024 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396028 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396030 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 02:59:26.398760 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396033 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396036 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396038 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396041 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396043 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396046 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396049 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396052 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396054 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396058 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396062 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396065 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396068 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396072 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396075 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396077 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396080 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396083 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396086 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 02:59:26.399257 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396469 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396474 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396477 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396481 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396484 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396488 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396491 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396494 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396497 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396500 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396503 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396506 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396509 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396511 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396514 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396516 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396520 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396523 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396526 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396528 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 02:59:26.399724 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396531 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396534 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396536 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396539 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396542 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396545 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396548 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396550 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396553 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396556 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396558 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396561 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396563 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396566 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396569 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396571 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396574 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396577 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396579 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 02:59:26.400223 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396583 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396587 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396590 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396592 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396595 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396598 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396603 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396606 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396609 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396612 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396615 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396619 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396623 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396626 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396629 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396631 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396634 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396636 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396639 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 02:59:26.400744 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396642 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396644 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396647 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396649 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396652 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396654 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396657 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396659 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396662 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396664 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396667 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396670 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396672 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396675 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396678 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396681 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396683 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396686 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396688 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396695 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 02:59:26.401215 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396697 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396700 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396703 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396705 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396708 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396711 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396714 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396717 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396720 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396722 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396725 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396727 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396730 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396732 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396735 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396737 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396740 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396742 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396745 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396747 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 02:59:26.401705 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.396750 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396824 2564 flags.go:64] FLAG: --address="0.0.0.0" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396841 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396848 2564 flags.go:64] FLAG: --anonymous-auth="true" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396852 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396857 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396860 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396865 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396870 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396873 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396876 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396881 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396885 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396888 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396891 2564 flags.go:64] FLAG: --cgroup-root="" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396894 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396897 2564 flags.go:64] FLAG: --client-ca-file="" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396900 2564 flags.go:64] FLAG: --cloud-provider="external" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396903 2564 flags.go:64] FLAG: --cluster-dns="[]" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396908 2564 flags.go:64] FLAG: --cluster-domain="" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396911 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396914 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396917 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396920 2564 flags.go:64] FLAG: --container-log-max-files="5" Jun 09 02:59:26.402191 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396924 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396927 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396930 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396933 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396936 2564 flags.go:64] FLAG: --contention-profiling="false" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396939 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396942 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396945 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396948 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396952 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396955 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396958 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396961 2564 flags.go:64] FLAG: --enable-load-reader="false" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396964 2564 flags.go:64] FLAG: --enable-server="true" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396967 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396971 2564 flags.go:64] FLAG: --event-burst="100" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396974 2564 flags.go:64] FLAG: --event-qps="50" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396977 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396980 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396984 2564 flags.go:64] FLAG: --eviction-hard="" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396988 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396992 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396995 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.396998 2564 flags.go:64] FLAG: --eviction-soft="" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397001 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 09 02:59:26.402834 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397003 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397006 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397010 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397013 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397015 2564 flags.go:64] FLAG: --fail-swap-on="true" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397019 2564 flags.go:64] FLAG: --feature-gates="" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397023 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397026 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397029 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397032 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397035 2564 flags.go:64] FLAG: --healthz-port="10248" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397038 2564 flags.go:64] FLAG: --help="false" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397041 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397044 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397046 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397049 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397052 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397056 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397059 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397061 2564 flags.go:64] FLAG: --image-service-endpoint="" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397065 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397067 2564 flags.go:64] FLAG: --kube-api-burst="100" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397071 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397074 2564 flags.go:64] FLAG: --kube-api-qps="50" Jun 09 02:59:26.404002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397077 2564 flags.go:64] FLAG: --kube-reserved="" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397080 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397085 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397090 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397092 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397095 2564 flags.go:64] FLAG: --lock-file="" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397098 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397101 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397104 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397110 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397113 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397115 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397118 2564 flags.go:64] FLAG: --logging-format="text" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397122 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397125 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397128 2564 flags.go:64] FLAG: --manifest-url="" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397131 2564 flags.go:64] FLAG: --manifest-url-header="" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397135 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397138 2564 flags.go:64] FLAG: --max-open-files="1000000" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397142 2564 flags.go:64] FLAG: --max-pods="110" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397145 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397148 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397151 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397155 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397158 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 09 02:59:26.405066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397161 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397164 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397170 2564 flags.go:64] FLAG: --node-status-max-images="50" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397173 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397176 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397179 2564 flags.go:64] FLAG: --pod-cidr="" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397183 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397188 2564 flags.go:64] FLAG: --pod-manifest-path="" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397191 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397194 2564 flags.go:64] FLAG: --pods-per-core="0" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397212 2564 flags.go:64] FLAG: --port="10250" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397215 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397218 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dcff69c57d39b5a0" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397221 2564 flags.go:64] FLAG: --qos-reserved="" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397224 2564 flags.go:64] FLAG: --read-only-port="10255" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397227 2564 flags.go:64] FLAG: --register-node="true" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397230 2564 flags.go:64] FLAG: --register-with-taints="" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397234 2564 flags.go:64] FLAG: --registry-burst="10" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397236 2564 flags.go:64] FLAG: --registry-qps="5" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397239 2564 flags.go:64] FLAG: --reserved-cpus="" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397242 2564 flags.go:64] FLAG: --reserved-memory="" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397246 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397249 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397252 2564 flags.go:64] FLAG: --rotate-certificates="false" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397255 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 09 02:59:26.406258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397259 2564 flags.go:64] FLAG: --runonce="false" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397261 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397265 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397268 2564 flags.go:64] FLAG: --seccomp-default="false" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397270 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397273 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397276 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397279 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397282 2564 flags.go:64] FLAG: --storage-driver-password="root" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397285 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397288 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397291 2564 flags.go:64] FLAG: --storage-driver-user="root" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397294 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397297 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397300 2564 flags.go:64] FLAG: --system-cgroups="" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397303 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397309 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397314 2564 flags.go:64] FLAG: --tls-cert-file="" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397317 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397321 2564 flags.go:64] FLAG: --tls-min-version="" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397324 2564 flags.go:64] FLAG: --tls-private-key-file="" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397327 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397330 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397333 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397336 2564 flags.go:64] FLAG: --v="2" Jun 09 02:59:26.407033 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397340 2564 flags.go:64] FLAG: --version="false" Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397344 2564 flags.go:64] FLAG: --vmodule="" Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397348 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397351 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397464 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397467 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397471 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397474 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397476 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397479 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397482 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397484 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397487 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397490 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397493 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397496 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397498 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397501 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397504 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397506 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 02:59:26.408038 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397509 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397511 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397514 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397517 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397523 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397526 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397529 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397531 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397534 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397536 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397539 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397541 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397544 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397547 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397549 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397552 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397554 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397557 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397560 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397563 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 02:59:26.408779 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397565 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397568 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397571 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397573 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397576 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397578 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397580 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397583 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397586 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397588 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397591 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397593 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397596 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397599 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397601 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397604 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397609 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397612 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397615 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397617 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 02:59:26.409425 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397620 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397622 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397625 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397628 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397631 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397635 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397638 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397642 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397645 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397648 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397651 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397653 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397656 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397659 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397662 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397665 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397667 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397670 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397672 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 02:59:26.409953 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397675 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397678 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397680 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397683 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397685 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397688 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397690 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397694 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397697 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397704 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397706 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397709 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397712 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397715 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397718 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397720 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397723 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397725 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397728 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397730 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 02:59:26.410557 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397733 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397736 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397738 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.397741 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.397746 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.405078 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.405095 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.405124 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.405133 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405234 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405242 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405248 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405254 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405259 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405263 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 02:59:26.411110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405267 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405271 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405275 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405280 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405284 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405291 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405297 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405303 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405307 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405312 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405318 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405322 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405326 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405331 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405335 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405339 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405343 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405348 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405352 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 02:59:26.411591 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405357 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405361 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405365 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405369 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405374 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405378 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405382 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405386 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405392 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405398 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405403 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405407 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405412 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405416 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405420 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405424 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405429 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405433 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405437 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405441 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 02:59:26.412049 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405447 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405451 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405456 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405460 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405464 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405468 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405473 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405476 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405481 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405485 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405489 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405494 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405498 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405502 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405507 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405511 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405516 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405520 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405526 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405533 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 02:59:26.412575 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405538 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405543 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405547 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405552 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405556 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405560 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405564 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405568 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405572 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405577 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405581 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405585 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405590 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405594 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405599 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405603 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405607 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405612 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405616 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 02:59:26.413296 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405622 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405626 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405633 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405638 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405643 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405647 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405652 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405656 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405661 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405666 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405670 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405674 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405679 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405683 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 02:59:26.413869 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405687 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.405695 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405876 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405884 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405888 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405893 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405897 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405902 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405906 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405911 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405915 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405919 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405923 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405928 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405932 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 02:59:26.414415 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405936 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405941 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405945 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405949 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405953 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405957 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405961 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405967 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405973 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405978 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405983 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405987 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405991 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.405996 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406000 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406004 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406009 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406013 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406017 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 02:59:26.414946 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406020 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406025 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406029 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406034 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406038 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406042 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406046 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406051 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406055 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406059 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406063 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406067 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406072 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406076 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406080 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406084 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406088 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406092 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406096 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 02:59:26.415586 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406100 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406104 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406108 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406112 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406117 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406121 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406125 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406132 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406138 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406143 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406147 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406152 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406157 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406162 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406166 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406170 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406175 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406179 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406183 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406187 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 02:59:26.416066 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406191 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406216 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406222 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406226 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406230 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406234 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406239 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406243 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406247 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406252 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406255 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406259 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406263 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406267 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406271 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406276 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406280 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406284 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406288 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406292 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 02:59:26.416572 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406297 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406301 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406308 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406313 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406319 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406323 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406328 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.406332 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.406339 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.406492 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.409158 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.410609 2564 server.go:1019] "Starting client certificate rotation" Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.410735 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 02:59:26.417064 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.410771 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 02:59:26.417418 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.415899 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 02:59:26.418577 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.418556 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 02:59:26.430288 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.430268 2564 log.go:25] "Validated CRI v1 runtime API" Jun 09 02:59:26.432956 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.432939 2564 log.go:25] "Validated CRI v1 image API" Jun 09 02:59:26.433794 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.433768 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 09 02:59:26.436009 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.435982 2564 fs.go:135] Filesystem UUIDs: map[56fd0b12-2d4d-46cd-900d-7f1ef1166470:/dev/nvme0n1p3 64b482f5-e5d6-4daf-bccc-60a9ff46b2bf:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 09 02:59:26.436082 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.436008 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 09 02:59:26.440957 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.440851 2564 manager.go:216] Machine: {Timestamp:2026-06-09 02:59:26.440378387 +0000 UTC m=+0.349892740 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3078408 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec261d30df050f775a5af3a856a97de9 SystemUUID:ec261d30-df05-0f77-5a5a-f3a856a97de9 BootID:639054ee-fcc5-4a91-97ef-fa51ccf4b75b Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:18:ef:8b:7b:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:18:ef:8b:7b:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:b7:da:9a:cd:36 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 09 02:59:26.440957 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.440950 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 09 02:59:26.441088 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.441063 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 09 02:59:26.441366 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.441347 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 02:59:26.442283 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.442255 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 09 02:59:26.442428 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.442285 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-143.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 09 02:59:26.443625 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.443614 2564 topology_manager.go:138] "Creating topology manager with none policy" Jun 09 02:59:26.443662 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.443628 2564 container_manager_linux.go:309] "Creating device plugin manager" Jun 09 02:59:26.443662 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.443636 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 02:59:26.444276 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.444265 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 02:59:26.445284 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.445275 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 09 02:59:26.451441 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.451427 2564 state_mem.go:36] "Initialized new in-memory state store" Jun 09 02:59:26.451570 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.451560 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 09 02:59:26.451613 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.451589 2564 kubelet.go:486] "Attempting to sync node with API server" Jun 09 02:59:26.451613 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.451603 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 09 02:59:26.451683 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.451619 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 09 02:59:26.451683 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.451651 2564 kubelet.go:398] "Adding apiserver pod source" Jun 09 02:59:26.451683 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.451681 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 09 02:59:26.452209 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.452179 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 02:59:26.452209 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.452210 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 02:59:26.455722 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.455701 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 09 02:59:26.457142 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.457126 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 09 02:59:26.457255 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.457165 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 09 02:59:26.460851 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460835 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460858 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460867 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460875 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460887 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460896 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460908 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460916 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460925 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460933 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 09 02:59:26.460941 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460945 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 09 02:59:26.461380 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460954 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 09 02:59:26.461380 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460988 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 09 02:59:26.461380 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.460997 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 09 02:59:26.462485 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.462464 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-js2qk" Jun 09 02:59:26.464316 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.464296 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-143.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 09 02:59:26.464405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.464365 2564 server.go:1268] "Started kubelet" Jun 09 02:59:26.464554 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.464529 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 09 02:59:26.464654 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.464566 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 09 02:59:26.464906 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.464879 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 09 02:59:26.464989 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.464907 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-143.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 09 02:59:26.464989 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.464963 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 09 02:59:26.465233 ip-10-0-130-143 systemd[1]: Started Kubernetes Kubelet. Jun 09 02:59:26.467085 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.467067 2564 server.go:310] "Adding debug handlers to kubelet server" Jun 09 02:59:26.467603 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.467582 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 09 02:59:26.469149 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.469131 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-js2qk" Jun 09 02:59:26.471757 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.471739 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 09 02:59:26.473023 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.472010 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-143.ec2.internal.18b74a62f153455f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-143.ec2.internal,UID:ip-10-0-130-143.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-143.ec2.internal,},FirstTimestamp:2026-06-09 02:59:26.464337247 +0000 UTC m=+0.373851609,LastTimestamp:2026-06-09 02:59:26.464337247 +0000 UTC m=+0.373851609,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-143.ec2.internal,}" Jun 09 02:59:26.474577 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.474562 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 09 02:59:26.474650 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.474565 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 09 02:59:26.474754 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.474739 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:26.476087 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.475248 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 09 02:59:26.476087 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.475278 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 09 02:59:26.476087 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.475297 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 09 02:59:26.476087 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.475423 2564 reconstruct.go:97] "Volume reconstruction finished" Jun 09 02:59:26.476087 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.475435 2564 reconciler.go:29] "Reconciler: start to sync state" Jun 09 02:59:26.476816 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.476797 2564 factory.go:55] Registering systemd factory Jun 09 02:59:26.476816 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.476818 2564 factory.go:223] Registration of the systemd container factory successfully Jun 09 02:59:26.477147 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.477133 2564 factory.go:153] Registering CRI-O factory Jun 09 02:59:26.477147 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.477146 2564 factory.go:223] Registration of the crio container factory successfully Jun 09 02:59:26.477298 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.477215 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 09 02:59:26.477298 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.477241 2564 factory.go:103] Registering Raw factory Jun 09 02:59:26.477298 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.477255 2564 manager.go:1196] Started watching for new ooms in manager Jun 09 02:59:26.477697 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.477686 2564 manager.go:349] Starting recovery of all containers Jun 09 02:59:26.482874 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.482822 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 02:59:26.485812 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.485788 2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-143.ec2.internal\" not found" node="ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.490345 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:26.490112 2564 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service/memory.swap.max": read /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service/memory.swap.max: no such device Jun 09 02:59:26.490574 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.490556 2564 manager.go:354] Recovery completed Jun 09 02:59:26.494713 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.494702 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 02:59:26.497298 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.497281 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientMemory" Jun 09 02:59:26.497378 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.497311 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 02:59:26.497378 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.497327 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientPID" Jun 09 02:59:26.497883 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.497867 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 09 02:59:26.497883 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.497877 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 02:59:26.498027 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.497879 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 09 02:59:26.498027 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.497943 2564 state_mem.go:36] "Initialized new in-memory state store" Jun 09 02:59:26.498592 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.498577 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientMemory" Jun 09 02:59:26.498661 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.498604 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 02:59:26.498661 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.498614 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientPID" Jun 09 02:59:26.499082 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.499069 2564 policy_none.go:49] "None policy: Start" Jun 09 02:59:26.499136 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.499089 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 09 02:59:26.499136 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.499102 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 09 02:59:26.500142 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.500129 2564 policy_none.go:47] "Start" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.539614 2564 manager.go:337] "Starting Device Plugin manager" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.539812 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.539826 2564 server.go:87] "Starting device plugin registration server" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.540115 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.540136 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.540256 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.540347 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.540376 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.540913 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.540995 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.542948 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.544135 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.544169 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.544188 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jun 09 02:59:26.545282 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.544237 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 09 02:59:26.546568 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.546549 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 02:59:26.641068 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.640987 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 02:59:26.642007 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.641970 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientMemory" Jun 09 02:59:26.642007 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.642011 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 02:59:26.642134 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.642021 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientPID" Jun 09 02:59:26.642134 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.642050 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.645237 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.645195 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal"] Jun 09 02:59:26.645292 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.645283 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 02:59:26.646755 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.646737 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientMemory" Jun 09 02:59:26.646845 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.646762 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 02:59:26.646845 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.646772 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientPID" Jun 09 02:59:26.647727 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.647713 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.647776 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.647734 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-143.ec2.internal\": node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:26.648043 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648032 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 02:59:26.648188 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648173 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.648243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648226 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 02:59:26.648785 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648767 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientMemory" Jun 09 02:59:26.648852 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648772 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientMemory" Jun 09 02:59:26.648852 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648816 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 02:59:26.648852 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648833 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientPID" Jun 09 02:59:26.648939 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648793 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 02:59:26.648939 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.648873 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientPID" Jun 09 02:59:26.649913 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.649895 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.650002 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.649926 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 02:59:26.650680 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.650660 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientMemory" Jun 09 02:59:26.650762 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.650694 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 02:59:26.650762 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.650710 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeHasSufficientPID" Jun 09 02:59:26.658546 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.658527 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:26.675907 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.675883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c95391064b04e0cb3e89407b59acb610-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal\" (UID: \"c95391064b04e0cb3e89407b59acb610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.675907 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.675916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff922de98c02d8203e86d43a866df755-config\") pod \"kube-apiserver-proxy-ip-10-0-130-143.ec2.internal\" (UID: \"ff922de98c02d8203e86d43a866df755\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.676066 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.675938 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c95391064b04e0cb3e89407b59acb610-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal\" (UID: \"c95391064b04e0cb3e89407b59acb610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.677147 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.677132 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-143.ec2.internal\" not found" node="ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.681489 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.681472 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-143.ec2.internal\" not found" node="ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.758983 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.758950 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:26.776314 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.776284 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c95391064b04e0cb3e89407b59acb610-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal\" (UID: \"c95391064b04e0cb3e89407b59acb610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.776448 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.776323 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff922de98c02d8203e86d43a866df755-config\") pod \"kube-apiserver-proxy-ip-10-0-130-143.ec2.internal\" (UID: \"ff922de98c02d8203e86d43a866df755\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.776448 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.776327 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c95391064b04e0cb3e89407b59acb610-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal\" (UID: \"c95391064b04e0cb3e89407b59acb610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.776448 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.776362 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c95391064b04e0cb3e89407b59acb610-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal\" (UID: \"c95391064b04e0cb3e89407b59acb610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.776448 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.776341 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c95391064b04e0cb3e89407b59acb610-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal\" (UID: \"c95391064b04e0cb3e89407b59acb610\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.776448 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.776378 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff922de98c02d8203e86d43a866df755-config\") pod \"kube-apiserver-proxy-ip-10-0-130-143.ec2.internal\" (UID: \"ff922de98c02d8203e86d43a866df755\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.859320 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.859275 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:26.960081 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:26.960007 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:26.981507 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.981476 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:26.985773 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:26.985743 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" Jun 09 02:59:27.060937 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:27.060893 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:27.161523 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:27.161494 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:27.262034 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:27.261966 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:27.362647 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:27.362618 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:27.410105 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.410074 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 09 02:59:27.410674 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.410312 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 02:59:27.410674 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.410310 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 02:59:27.463645 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:27.463620 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:27.469900 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:27.469841 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff922de98c02d8203e86d43a866df755.slice/crio-c47279355ea701663cbf4d9001731fa16e6c3fc5f4d96b95dbd76641ad3695eb WatchSource:0}: Error finding container c47279355ea701663cbf4d9001731fa16e6c3fc5f4d96b95dbd76641ad3695eb: Status 404 returned error can't find the container with id c47279355ea701663cbf4d9001731fa16e6c3fc5f4d96b95dbd76641ad3695eb Jun 09 02:59:27.470260 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:27.470233 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95391064b04e0cb3e89407b59acb610.slice/crio-618e88f7f7cd559b900cc80320173a23bb05d6eb9c71f59b0a384195a3ffce69 WatchSource:0}: Error finding container 618e88f7f7cd559b900cc80320173a23bb05d6eb9c71f59b0a384195a3ffce69: Status 404 returned error can't find the container with id 618e88f7f7cd559b900cc80320173a23bb05d6eb9c71f59b0a384195a3ffce69 Jun 09 02:59:27.474480 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.474416 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-08 02:54:26 +0000 UTC" deadline="2027-11-20 09:02:40.781635233 +0000 UTC" Jun 09 02:59:27.474565 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.474480 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12702h3m13.307159257s" Jun 09 02:59:27.475459 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.475444 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 02:59:27.547780 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.547664 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" event={"ID":"c95391064b04e0cb3e89407b59acb610","Type":"ContainerStarted","Data":"618e88f7f7cd559b900cc80320173a23bb05d6eb9c71f59b0a384195a3ffce69"} Jun 09 02:59:27.548627 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.548605 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" event={"ID":"ff922de98c02d8203e86d43a866df755","Type":"ContainerStarted","Data":"c47279355ea701663cbf4d9001731fa16e6c3fc5f4d96b95dbd76641ad3695eb"} Jun 09 02:59:27.563714 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:27.563687 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:27.664237 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:27.664190 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:27.690796 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.690767 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 02:59:27.765382 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:27.765341 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-143.ec2.internal\" not found" Jun 09 02:59:27.806942 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.806885 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 02:59:27.875710 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.875667 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" Jun 09 02:59:27.885921 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.885893 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 02:59:27.886795 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.886773 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" Jun 09 02:59:27.894056 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:27.894019 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 02:59:28.415991 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.415906 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 02:59:28.453368 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.453341 2564 apiserver.go:52] "Watching apiserver" Jun 09 02:59:28.461350 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.461321 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 09 02:59:28.465156 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.465122 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn","openshift-dns/node-resolver-dg64b","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal","openshift-multus/multus-r2p87","openshift-network-operator/iptables-alerter-wp5c7","kube-system/konnectivity-agent-xhqhk","openshift-cluster-node-tuning-operator/tuned-rfdn2","openshift-image-registry/node-ca-vww9x","openshift-multus/multus-additional-cni-plugins-789lw","openshift-multus/network-metrics-daemon-l65ml","openshift-network-diagnostics/network-check-target-dntqr","openshift-ovn-kubernetes/ovnkube-node-g7wpr","kube-system/global-pull-secret-syncer-mjbdx","kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal"] Jun 09 02:59:28.466775 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.466752 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.467886 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.467860 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.467991 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.467933 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:28.468794 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.468769 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j65vs\"" Jun 09 02:59:28.468902 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.468773 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 09 02:59:28.468902 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.468829 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 09 02:59:28.469042 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.469028 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.470161 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.470139 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.471145 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.471122 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 09 02:59:28.471280 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.471264 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 09 02:59:28.472034 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.471665 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vtq82\"" Jun 09 02:59:28.472034 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.471712 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 09 02:59:28.472034 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.471900 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 09 02:59:28.472269 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.472179 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 09 02:59:28.472269 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.472243 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ldd8v\"" Jun 09 02:59:28.472365 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.472349 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 09 02:59:28.473835 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.472829 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 09 02:59:28.474258 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.474236 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.474811 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.474747 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 09 02:59:28.476147 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.475600 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:28.477293 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.477096 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 09 02:59:28.477293 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.477160 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 09 02:59:28.477293 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.477252 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 09 02:59:28.477293 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.477277 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v92fb\"" Jun 09 02:59:28.478060 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.477675 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.478060 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.477770 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cn77d\"" Jun 09 02:59:28.478060 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.477781 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 09 02:59:28.478060 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.477993 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 09 02:59:28.479065 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.479046 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.479809 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.479572 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 09 02:59:28.479809 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.479661 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sl2vk\"" Jun 09 02:59:28.479809 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.479743 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 09 02:59:28.480596 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.480569 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 09 02:59:28.481045 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.480763 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7t9c4\"" Jun 09 02:59:28.481045 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.480798 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 09 02:59:28.481045 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.480911 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.481045 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.480934 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 09 02:59:28.481851 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.481832 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:28.481936 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.481907 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:28.482438 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.482416 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-st8w2\"" Jun 09 02:59:28.482856 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.482838 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 09 02:59:28.482856 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.482849 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 09 02:59:28.483486 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.483379 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:28.483486 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.483448 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:28.484956 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.484936 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.485885 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-var-lib-kubelet\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.485885 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485716 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2fa6373-a391-4405-8a41-fb03ae52a042-host\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.485885 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485745 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-netns\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.485885 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-hostroot\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.485885 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485790 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-modprobe-d\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.485885 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485816 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-systemd\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.485885 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-sys\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.485885 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485878 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e2fa6373-a391-4405-8a41-fb03ae52a042-serviceca\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-conf-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485963 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a15392b-3ee8-4387-a5c4-83617bea6bda-host-slash\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.485989 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-kubernetes\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486012 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-lib-modules\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486037 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c4646af-5134-4c7f-af61-b304e3a02a71-tmp\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486075 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2lmv\" (UniqueName: \"kubernetes.io/projected/e2fa6373-a391-4405-8a41-fb03ae52a042-kube-api-access-c2lmv\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486108 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rqng\" (UniqueName: \"kubernetes.io/projected/0a15392b-3ee8-4387-a5c4-83617bea6bda-kube-api-access-6rqng\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486134 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2adb943-7cd5-43df-bdce-3309518d60c5-agent-certs\") pod \"konnectivity-agent-xhqhk\" (UID: \"a2adb943-7cd5-43df-bdce-3309518d60c5\") " pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486156 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysctl-d\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-run\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486184 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-tuned\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486218 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-dbus\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486317 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-system-cni-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486405 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486403 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-k8s-cni-cncf-io\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysconfig\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486447 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysctl-conf\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486462 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-cni-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486476 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-cnibin\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486491 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dc94e9a-d865-4694-9a6a-41f628b714de-cni-binary-copy\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486519 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-cni-bin\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486533 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-kubelet\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486548 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-multus-certs\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486562 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgg4k\" (UniqueName: \"kubernetes.io/projected/6c4646af-5134-4c7f-af61-b304e3a02a71-kube-api-access-xgg4k\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486577 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-kubelet-config\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486594 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486608 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-os-release\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486622 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-etc-kubernetes\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486660 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-host\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486683 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-socket-dir-parent\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486723 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-cni-multus\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486809 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hc2sg\"" Jun 09 02:59:28.486951 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486812 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-daemon-config\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgzvs\" (UniqueName: \"kubernetes.io/projected/1dc94e9a-d865-4694-9a6a-41f628b714de-kube-api-access-bgzvs\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a15392b-3ee8-4387-a5c4-83617bea6bda-iptables-alerter-script\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.486995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2adb943-7cd5-43df-bdce-3309518d60c5-konnectivity-ca\") pod \"konnectivity-agent-xhqhk\" (UID: \"a2adb943-7cd5-43df-bdce-3309518d60c5\") " pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.487063 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.487075 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.487142 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.487175 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 09 02:59:28.487778 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.487462 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 09 02:59:28.491671 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.491652 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 02:59:28.502543 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.502523 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ls92x" Jun 09 02:59:28.510042 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.510024 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ls92x" Jun 09 02:59:28.577140 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.577111 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 09 02:59:28.587681 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587656 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58bw4\" (UniqueName: \"kubernetes.io/projected/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-kube-api-access-58bw4\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587697 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-ovn\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-lib-modules\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c4646af-5134-4c7f-af61-b304e3a02a71-tmp\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rqng\" (UniqueName: \"kubernetes.io/projected/0a15392b-3ee8-4387-a5c4-83617bea6bda-kube-api-access-6rqng\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-kubelet\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587825 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-var-lib-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587846 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-log-socket\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.587859 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587861 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysctl-d\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-dbus\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587896 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-k8s-cni-cncf-io\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587936 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b73b4444-c489-4e85-aabd-b69d0c1553b3-hosts-file\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-device-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.587999 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-ovnkube-config\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-env-overrides\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588032 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysconfig\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588047 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-cnibin\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588061 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dc94e9a-d865-4694-9a6a-41f628b714de-cni-binary-copy\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-cni-bin\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588101 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-kubelet\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588115 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-multus-certs\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588133 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-etc-kubernetes\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-registration-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588173 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-kubelet-config\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588213 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.588357 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588242 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-k8s-cni-cncf-io\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588319 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-os-release\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588368 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-kubelet-config\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588454 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-lib-modules\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.588484 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588540 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-cni-bin\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.588556 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret podName:125f4810-5dbf-4e8f-9eb0-ecd8e22564d4 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:29.088523158 +0000 UTC m=+2.998037496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret") pod "global-pull-secret-syncer-mjbdx" (UID: "125f4810-5dbf-4e8f-9eb0-ecd8e22564d4") : object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588587 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysconfig\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588624 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-cnibin\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588678 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysctl-d\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-os-release\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588775 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588795 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-systemd\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-run-ovn-kubernetes\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588843 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-cni-bin\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588857 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-etc-kubernetes\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588872 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-cni-netd\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588895 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-dbus\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:28.589339 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588919 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/db8ebec1-7194-4799-a74b-bc212211f99e-ovn-node-metrics-cert\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588934 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-multus-certs\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588939 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-kubelet\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.588980 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-socket-dir-parent\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589032 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgzvs\" (UniqueName: \"kubernetes.io/projected/1dc94e9a-d865-4694-9a6a-41f628b714de-kube-api-access-bgzvs\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589043 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-socket-dir-parent\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589076 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dc94e9a-d865-4694-9a6a-41f628b714de-cni-binary-copy\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589073 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqmzn\" (UniqueName: \"kubernetes.io/projected/b73b4444-c489-4e85-aabd-b69d0c1553b3-kube-api-access-rqmzn\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589227 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-node-log\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-ovnkube-script-lib\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589387 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-os-release\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-tuning-conf-dir\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589461 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-var-lib-kubelet\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589498 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2fa6373-a391-4405-8a41-fb03ae52a042-host\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589535 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589540 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-var-lib-kubelet\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589567 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-slash\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.590081 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589581 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2fa6373-a391-4405-8a41-fb03ae52a042-host\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589610 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589637 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-netns\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589681 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kl4k\" (UniqueName: \"kubernetes.io/projected/db8ebec1-7194-4799-a74b-bc212211f99e-kube-api-access-9kl4k\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589699 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-run-netns\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589744 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-cni-binary-copy\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589776 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589814 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-systemd\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589850 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-sys\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589908 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-systemd\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589920 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-sys\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589940 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.589976 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-kubernetes\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2lmv\" (UniqueName: \"kubernetes.io/projected/e2fa6373-a391-4405-8a41-fb03ae52a042-kube-api-access-c2lmv\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590056 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2adb943-7cd5-43df-bdce-3309518d60c5-agent-certs\") pod \"konnectivity-agent-xhqhk\" (UID: \"a2adb943-7cd5-43df-bdce-3309518d60c5\") " pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-run\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590111 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-tuned\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.590880 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590133 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-system-cni-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590160 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a15392b-3ee8-4387-a5c4-83617bea6bda-iptables-alerter-script\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-cnibin\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysctl-conf\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590288 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-system-cni-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590307 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-cni-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg4k\" (UniqueName: \"kubernetes.io/projected/6c4646af-5134-4c7f-af61-b304e3a02a71-kube-api-access-xgg4k\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590361 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-system-cni-dir\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590387 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-host\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590453 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-cni-multus\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590537 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-run\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590549 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-cni-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590570 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-sysctl-conf\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590599 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-host\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590621 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-kubernetes\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590633 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-daemon-config\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590661 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-sys-fs\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.591679 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fnph\" (UniqueName: \"kubernetes.io/projected/9df3f892-882c-409c-a7ac-44e81fe9c003-kube-api-access-9fnph\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590707 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-host-var-lib-cni-multus\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590736 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blmzv\" (UniqueName: \"kubernetes.io/projected/348a74fc-1fea-4472-ae4a-d26df51b1993-kube-api-access-blmzv\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2adb943-7cd5-43df-bdce-3309518d60c5-konnectivity-ca\") pod \"konnectivity-agent-xhqhk\" (UID: \"a2adb943-7cd5-43df-bdce-3309518d60c5\") " pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590816 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-run-netns\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590914 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-hostroot\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590936 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73b4444-c489-4e85-aabd-b69d0c1553b3-tmp-dir\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.590958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-systemd-units\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591045 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-etc-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591090 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-modprobe-d\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591137 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a15392b-3ee8-4387-a5c4-83617bea6bda-iptables-alerter-script\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591152 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e2fa6373-a391-4405-8a41-fb03ae52a042-serviceca\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591187 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-conf-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591233 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a15392b-3ee8-4387-a5c4-83617bea6bda-host-slash\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-socket-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.592477 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591285 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-modprobe-d\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.593260 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591330 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-hostroot\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.593260 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591342 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-daemon-config\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.593260 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591369 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a15392b-3ee8-4387-a5c4-83617bea6bda-host-slash\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.593260 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591416 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dc94e9a-d865-4694-9a6a-41f628b714de-multus-conf-dir\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.593260 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591481 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2adb943-7cd5-43df-bdce-3309518d60c5-konnectivity-ca\") pod \"konnectivity-agent-xhqhk\" (UID: \"a2adb943-7cd5-43df-bdce-3309518d60c5\") " pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:28.593260 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.591714 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e2fa6373-a391-4405-8a41-fb03ae52a042-serviceca\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.593771 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.593736 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c4646af-5134-4c7f-af61-b304e3a02a71-tmp\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.593856 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.593784 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c4646af-5134-4c7f-af61-b304e3a02a71-etc-tuned\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.595018 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.594998 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2adb943-7cd5-43df-bdce-3309518d60c5-agent-certs\") pod \"konnectivity-agent-xhqhk\" (UID: \"a2adb943-7cd5-43df-bdce-3309518d60c5\") " pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:28.600403 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.600380 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rqng\" (UniqueName: \"kubernetes.io/projected/0a15392b-3ee8-4387-a5c4-83617bea6bda-kube-api-access-6rqng\") pod \"iptables-alerter-wp5c7\" (UID: \"0a15392b-3ee8-4387-a5c4-83617bea6bda\") " pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.600525 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.600431 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgg4k\" (UniqueName: \"kubernetes.io/projected/6c4646af-5134-4c7f-af61-b304e3a02a71-kube-api-access-xgg4k\") pod \"tuned-rfdn2\" (UID: \"6c4646af-5134-4c7f-af61-b304e3a02a71\") " pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.600525 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.600464 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2lmv\" (UniqueName: \"kubernetes.io/projected/e2fa6373-a391-4405-8a41-fb03ae52a042-kube-api-access-c2lmv\") pod \"node-ca-vww9x\" (UID: \"e2fa6373-a391-4405-8a41-fb03ae52a042\") " pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.600704 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.600680 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgzvs\" (UniqueName: \"kubernetes.io/projected/1dc94e9a-d865-4694-9a6a-41f628b714de-kube-api-access-bgzvs\") pod \"multus-r2p87\" (UID: \"1dc94e9a-d865-4694-9a6a-41f628b714de\") " pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.692025 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.691945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-registration-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.692025 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.691989 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-systemd\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692025 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692009 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-run-ovn-kubernetes\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-cni-bin\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-cni-netd\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692078 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-systemd\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692086 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-registration-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692084 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/db8ebec1-7194-4799-a74b-bc212211f99e-ovn-node-metrics-cert\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692143 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-cni-netd\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692166 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-run-ovn-kubernetes\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692174 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-cni-bin\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692215 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqmzn\" (UniqueName: \"kubernetes.io/projected/b73b4444-c489-4e85-aabd-b69d0c1553b3-kube-api-access-rqmzn\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-node-log\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692277 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-ovnkube-script-lib\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692301 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-os-release\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692303 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692309 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-node-log\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-tuning-conf-dir\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692350 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692396 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-os-release\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-slash\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692464 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.692481 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692492 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-slash\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692493 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kl4k\" (UniqueName: \"kubernetes.io/projected/db8ebec1-7194-4799-a74b-bc212211f99e-kube-api-access-9kl4k\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.692531 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:29.192512896 +0000 UTC m=+3.102027234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692508 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-tuning-conf-dir\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692549 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-cni-binary-copy\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692600 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692637 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692670 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-cnibin\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692698 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-system-cni-dir\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.692898 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-sys-fs\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fnph\" (UniqueName: \"kubernetes.io/projected/9df3f892-882c-409c-a7ac-44e81fe9c003-kube-api-access-9fnph\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blmzv\" (UniqueName: \"kubernetes.io/projected/348a74fc-1fea-4472-ae4a-d26df51b1993-kube-api-access-blmzv\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692807 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692828 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-sys-fs\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692828 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-ovnkube-script-lib\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692866 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-cnibin\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692872 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/348a74fc-1fea-4472-ae4a-d26df51b1993-system-cni-dir\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692878 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692905 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-run-netns\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692922 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692939 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73b4444-c489-4e85-aabd-b69d0c1553b3-tmp-dir\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692958 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-systemd-units\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692979 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-run-netns\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.692981 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-etc-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-socket-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693037 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-etc-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.693691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693044 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58bw4\" (UniqueName: \"kubernetes.io/projected/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-kube-api-access-58bw4\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-ovn\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693074 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693081 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-systemd-units\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693096 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693121 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693153 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-kubelet\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693165 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-socket-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693192 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-run-ovn\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693157 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-cni-binary-copy\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693230 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693232 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-var-lib-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693264 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-log-socket\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693268 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-var-lib-openvswitch\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693292 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b73b4444-c489-4e85-aabd-b69d0c1553b3-hosts-file\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693308 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-host-kubelet\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.694547 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693260 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73b4444-c489-4e85-aabd-b69d0c1553b3-tmp-dir\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-device-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693332 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/db8ebec1-7194-4799-a74b-bc212211f99e-log-socket\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-ovnkube-config\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693370 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693377 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b73b4444-c489-4e85-aabd-b69d0c1553b3-hosts-file\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693392 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-env-overrides\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693415 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-device-dir\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693741 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-ovnkube-config\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.693745 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/db8ebec1-7194-4799-a74b-bc212211f99e-env-overrides\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.694176 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/348a74fc-1fea-4472-ae4a-d26df51b1993-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.695243 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.694883 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/db8ebec1-7194-4799-a74b-bc212211f99e-ovn-node-metrics-cert\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.698148 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.698130 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 02:59:28.702354 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.702333 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 02:59:28.702354 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.702354 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 02:59:28.702500 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.702365 2564 projected.go:196] Error preparing data for projected volume kube-api-access-kszw6 for pod openshift-network-diagnostics/network-check-target-dntqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:28.702500 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:28.702432 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6 podName:8df8eb28-db42-4b67-845f-604b94c50370 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:29.202417232 +0000 UTC m=+3.111931557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kszw6" (UniqueName: "kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6") pod "network-check-target-dntqr" (UID: "8df8eb28-db42-4b67-845f-604b94c50370") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:28.703830 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.703805 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqmzn\" (UniqueName: \"kubernetes.io/projected/b73b4444-c489-4e85-aabd-b69d0c1553b3-kube-api-access-rqmzn\") pod \"node-resolver-dg64b\" (UID: \"b73b4444-c489-4e85-aabd-b69d0c1553b3\") " pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.705252 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.705230 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fnph\" (UniqueName: \"kubernetes.io/projected/9df3f892-882c-409c-a7ac-44e81fe9c003-kube-api-access-9fnph\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:28.705343 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.705235 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58bw4\" (UniqueName: \"kubernetes.io/projected/ac0436e3-9fbb-4ed4-a153-1fda49d2d527-kube-api-access-58bw4\") pod \"aws-ebs-csi-driver-node-wqcfn\" (UID: \"ac0436e3-9fbb-4ed4-a153-1fda49d2d527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.705938 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.705917 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blmzv\" (UniqueName: \"kubernetes.io/projected/348a74fc-1fea-4472-ae4a-d26df51b1993-kube-api-access-blmzv\") pod \"multus-additional-cni-plugins-789lw\" (UID: \"348a74fc-1fea-4472-ae4a-d26df51b1993\") " pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.706286 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.706263 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kl4k\" (UniqueName: \"kubernetes.io/projected/db8ebec1-7194-4799-a74b-bc212211f99e-kube-api-access-9kl4k\") pod \"ovnkube-node-g7wpr\" (UID: \"db8ebec1-7194-4799-a74b-bc212211f99e\") " pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:28.781085 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.781045 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" Jun 09 02:59:28.789426 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.789402 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vww9x" Jun 09 02:59:28.799064 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.799042 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r2p87" Jun 09 02:59:28.805975 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.805953 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wp5c7" Jun 09 02:59:28.813637 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.813617 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:28.821164 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.821148 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-789lw" Jun 09 02:59:28.830231 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.830212 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" Jun 09 02:59:28.836488 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.836469 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dg64b" Jun 09 02:59:28.842131 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:28.842113 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:29.096566 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.096481 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:29.096709 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.096607 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:29.096709 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.096656 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret podName:125f4810-5dbf-4e8f-9eb0-ecd8e22564d4 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:30.096642042 +0000 UTC m=+4.006156368 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret") pod "global-pull-secret-syncer-mjbdx" (UID: "125f4810-5dbf-4e8f-9eb0-ecd8e22564d4") : object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:29.197531 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.197495 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:29.197682 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.197647 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:29.197761 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.197702 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:30.19768832 +0000 UTC m=+4.107202648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:29.278512 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.278482 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a15392b_3ee8_4387_a5c4_83617bea6bda.slice/crio-5bca9d3c6a40c992c7d1b352bea3f29b174c1203e2b21fb528f4e2a39ed72189 WatchSource:0}: Error finding container 5bca9d3c6a40c992c7d1b352bea3f29b174c1203e2b21fb528f4e2a39ed72189: Status 404 returned error can't find the container with id 5bca9d3c6a40c992c7d1b352bea3f29b174c1203e2b21fb528f4e2a39ed72189 Jun 09 02:59:29.282581 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.282532 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73b4444_c489_4e85_aabd_b69d0c1553b3.slice/crio-33ab8a06868484a98a1af9500f30303fe951c69f5523038e7d344431af2be31a WatchSource:0}: Error finding container 33ab8a06868484a98a1af9500f30303fe951c69f5523038e7d344431af2be31a: Status 404 returned error can't find the container with id 33ab8a06868484a98a1af9500f30303fe951c69f5523038e7d344431af2be31a Jun 09 02:59:29.286110 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.286085 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc94e9a_d865_4694_9a6a_41f628b714de.slice/crio-d94cc85074e808bc9cae7a598202079435a19c89bbccfa5f930f3b11c83c75b2 WatchSource:0}: Error finding container d94cc85074e808bc9cae7a598202079435a19c89bbccfa5f930f3b11c83c75b2: Status 404 returned error can't find the container with id d94cc85074e808bc9cae7a598202079435a19c89bbccfa5f930f3b11c83c75b2 Jun 09 02:59:29.286823 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.286799 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2adb943_7cd5_43df_bdce_3309518d60c5.slice/crio-f54caac4888540040aa75926409cad0183b1e2a9d86cfddfd0002349d528de32 WatchSource:0}: Error finding container f54caac4888540040aa75926409cad0183b1e2a9d86cfddfd0002349d528de32: Status 404 returned error can't find the container with id f54caac4888540040aa75926409cad0183b1e2a9d86cfddfd0002349d528de32 Jun 09 02:59:29.288056 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.288031 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb8ebec1_7194_4799_a74b_bc212211f99e.slice/crio-b4094fed95119bb87a8da3ccdfb01ef2482e4b41e194d68d7ae5cf6ea345f009 WatchSource:0}: Error finding container b4094fed95119bb87a8da3ccdfb01ef2482e4b41e194d68d7ae5cf6ea345f009: Status 404 returned error can't find the container with id b4094fed95119bb87a8da3ccdfb01ef2482e4b41e194d68d7ae5cf6ea345f009 Jun 09 02:59:29.288918 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.288831 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0436e3_9fbb_4ed4_a153_1fda49d2d527.slice/crio-561c48a9d251ea94842a4abc3f11a8aa1d5cd7c6f64a42a55ffea3235fb7ec08 WatchSource:0}: Error finding container 561c48a9d251ea94842a4abc3f11a8aa1d5cd7c6f64a42a55ffea3235fb7ec08: Status 404 returned error can't find the container with id 561c48a9d251ea94842a4abc3f11a8aa1d5cd7c6f64a42a55ffea3235fb7ec08 Jun 09 02:59:29.290072 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.290049 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod348a74fc_1fea_4472_ae4a_d26df51b1993.slice/crio-81746cfb63134d2eee6220798d34212de7c289f59eef2e9088613f4ceedc9351 WatchSource:0}: Error finding container 81746cfb63134d2eee6220798d34212de7c289f59eef2e9088613f4ceedc9351: Status 404 returned error can't find the container with id 81746cfb63134d2eee6220798d34212de7c289f59eef2e9088613f4ceedc9351 Jun 09 02:59:29.291671 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.291300 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4646af_5134_4c7f_af61_b304e3a02a71.slice/crio-84916c12250cc8b9e32779c1e3d3799cd0a67c8c956bbadcf870a410f3c0a689 WatchSource:0}: Error finding container 84916c12250cc8b9e32779c1e3d3799cd0a67c8c956bbadcf870a410f3c0a689: Status 404 returned error can't find the container with id 84916c12250cc8b9e32779c1e3d3799cd0a67c8c956bbadcf870a410f3c0a689 Jun 09 02:59:29.293268 ip-10-0-130-143 kubenswrapper[2564]: W0609 02:59:29.293139 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fa6373_a391_4405_8a41_fb03ae52a042.slice/crio-0a0040c0758f5a307b0052054c6512df15cd3e42ddc5def81368104d3c693eae WatchSource:0}: Error finding container 0a0040c0758f5a307b0052054c6512df15cd3e42ddc5def81368104d3c693eae: Status 404 returned error can't find the container with id 0a0040c0758f5a307b0052054c6512df15cd3e42ddc5def81368104d3c693eae Jun 09 02:59:29.298702 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.298683 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:29.298796 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.298783 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 02:59:29.298835 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.298803 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 02:59:29.298835 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.298813 2564 projected.go:196] Error preparing data for projected volume kube-api-access-kszw6 for pod openshift-network-diagnostics/network-check-target-dntqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:29.298940 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.298854 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6 podName:8df8eb28-db42-4b67-845f-604b94c50370 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:30.298838765 +0000 UTC m=+4.208353090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kszw6" (UniqueName: "kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6") pod "network-check-target-dntqr" (UID: "8df8eb28-db42-4b67-845f-604b94c50370") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:29.512940 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.512761 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 02:54:28 +0000 UTC" deadline="2028-02-22 09:50:02.278635774 +0000 UTC" Jun 09 02:59:29.512940 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.512938 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14958h50m32.765700502s" Jun 09 02:59:29.544609 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.544587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:29.544704 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:29.544685 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:29.552825 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.552791 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"b4094fed95119bb87a8da3ccdfb01ef2482e4b41e194d68d7ae5cf6ea345f009"} Jun 09 02:59:29.553785 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.553765 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" event={"ID":"ac0436e3-9fbb-4ed4-a153-1fda49d2d527","Type":"ContainerStarted","Data":"561c48a9d251ea94842a4abc3f11a8aa1d5cd7c6f64a42a55ffea3235fb7ec08"} Jun 09 02:59:29.554821 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.554799 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xhqhk" event={"ID":"a2adb943-7cd5-43df-bdce-3309518d60c5","Type":"ContainerStarted","Data":"f54caac4888540040aa75926409cad0183b1e2a9d86cfddfd0002349d528de32"} Jun 09 02:59:29.555789 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.555772 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wp5c7" event={"ID":"0a15392b-3ee8-4387-a5c4-83617bea6bda","Type":"ContainerStarted","Data":"5bca9d3c6a40c992c7d1b352bea3f29b174c1203e2b21fb528f4e2a39ed72189"} Jun 09 02:59:29.557589 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.557567 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" event={"ID":"ff922de98c02d8203e86d43a866df755","Type":"ContainerStarted","Data":"4d4824d95af7afc2f73609372309d3820779c85c7a50cac4551d33c4d4796699"} Jun 09 02:59:29.558692 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.558672 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vww9x" event={"ID":"e2fa6373-a391-4405-8a41-fb03ae52a042","Type":"ContainerStarted","Data":"0a0040c0758f5a307b0052054c6512df15cd3e42ddc5def81368104d3c693eae"} Jun 09 02:59:29.559791 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.559772 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" event={"ID":"6c4646af-5134-4c7f-af61-b304e3a02a71","Type":"ContainerStarted","Data":"84916c12250cc8b9e32779c1e3d3799cd0a67c8c956bbadcf870a410f3c0a689"} Jun 09 02:59:29.560872 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.560826 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-789lw" event={"ID":"348a74fc-1fea-4472-ae4a-d26df51b1993","Type":"ContainerStarted","Data":"81746cfb63134d2eee6220798d34212de7c289f59eef2e9088613f4ceedc9351"} Jun 09 02:59:29.561790 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.561772 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r2p87" event={"ID":"1dc94e9a-d865-4694-9a6a-41f628b714de","Type":"ContainerStarted","Data":"d94cc85074e808bc9cae7a598202079435a19c89bbccfa5f930f3b11c83c75b2"} Jun 09 02:59:29.562735 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.562716 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dg64b" event={"ID":"b73b4444-c489-4e85-aabd-b69d0c1553b3","Type":"ContainerStarted","Data":"33ab8a06868484a98a1af9500f30303fe951c69f5523038e7d344431af2be31a"} Jun 09 02:59:29.564166 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.564146 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" event={"ID":"c95391064b04e0cb3e89407b59acb610","Type":"ContainerStarted","Data":"df1b56796f5ee39e2d3977e7f01c5aeb7a0ff317f1d0b0f5a2dc919df4fe9755"} Jun 09 02:59:29.569189 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:29.569153 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-143.ec2.internal" podStartSLOduration=2.569142957 podStartE2EDuration="2.569142957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:29.568655899 +0000 UTC m=+3.478170247" watchObservedRunningTime="2026-06-09 02:59:29.569142957 +0000 UTC m=+3.478657304" Jun 09 02:59:30.103546 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.103473 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:30.103735 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.103698 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:30.103798 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.103762 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret podName:125f4810-5dbf-4e8f-9eb0-ecd8e22564d4 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:32.103744995 +0000 UTC m=+6.013259322 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret") pod "global-pull-secret-syncer-mjbdx" (UID: "125f4810-5dbf-4e8f-9eb0-ecd8e22564d4") : object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:30.204680 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.204643 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:30.204856 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.204806 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:30.204926 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.204867 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:32.204849378 +0000 UTC m=+6.114363703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:30.305631 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.305596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:30.305791 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.305778 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 02:59:30.305845 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.305798 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 02:59:30.305845 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.305812 2564 projected.go:196] Error preparing data for projected volume kube-api-access-kszw6 for pod openshift-network-diagnostics/network-check-target-dntqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:30.305952 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.305867 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6 podName:8df8eb28-db42-4b67-845f-604b94c50370 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:32.305849435 +0000 UTC m=+6.215363765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kszw6" (UniqueName: "kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6") pod "network-check-target-dntqr" (UID: "8df8eb28-db42-4b67-845f-604b94c50370") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:30.513570 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.513455 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 02:54:28 +0000 UTC" deadline="2028-02-24 06:28:31.838695124 +0000 UTC" Jun 09 02:59:30.513570 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.513513 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15003h29m1.325186719s" Jun 09 02:59:30.545325 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.545299 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:30.545457 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.545420 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:30.545821 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.545799 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:30.545918 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:30.545894 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:30.583736 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.583695 2564 generic.go:356] "Generic (PLEG): container finished" podID="c95391064b04e0cb3e89407b59acb610" containerID="df1b56796f5ee39e2d3977e7f01c5aeb7a0ff317f1d0b0f5a2dc919df4fe9755" exitCode=0 Jun 09 02:59:30.584643 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:30.584616 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" event={"ID":"c95391064b04e0cb3e89407b59acb610","Type":"ContainerDied","Data":"df1b56796f5ee39e2d3977e7f01c5aeb7a0ff317f1d0b0f5a2dc919df4fe9755"} Jun 09 02:59:31.544957 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:31.544929 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:31.545423 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:31.545067 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:31.592407 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:31.592373 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" event={"ID":"c95391064b04e0cb3e89407b59acb610","Type":"ContainerStarted","Data":"520fa91840b50d835bcc7d37df0effb988aef58eeaa9f65a7f8a200105f44397"} Jun 09 02:59:31.607657 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:31.607566 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-143.ec2.internal" podStartSLOduration=4.607547812 podStartE2EDuration="4.607547812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:31.607523385 +0000 UTC m=+5.517037733" watchObservedRunningTime="2026-06-09 02:59:31.607547812 +0000 UTC m=+5.517062164" Jun 09 02:59:32.126109 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:32.125539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:32.126109 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.125705 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:32.126109 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.125767 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret podName:125f4810-5dbf-4e8f-9eb0-ecd8e22564d4 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:36.125749251 +0000 UTC m=+10.035263578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret") pod "global-pull-secret-syncer-mjbdx" (UID: "125f4810-5dbf-4e8f-9eb0-ecd8e22564d4") : object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:32.226295 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:32.226257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:32.226486 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.226468 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:32.226562 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.226544 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:36.226524922 +0000 UTC m=+10.136039249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:32.328015 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:32.327625 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:32.328015 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.327839 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 02:59:32.328015 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.327858 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 02:59:32.328015 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.327871 2564 projected.go:196] Error preparing data for projected volume kube-api-access-kszw6 for pod openshift-network-diagnostics/network-check-target-dntqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:32.328015 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.327928 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6 podName:8df8eb28-db42-4b67-845f-604b94c50370 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:36.327907806 +0000 UTC m=+10.237422136 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kszw6" (UniqueName: "kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6") pod "network-check-target-dntqr" (UID: "8df8eb28-db42-4b67-845f-604b94c50370") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:32.546753 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:32.544736 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:32.546753 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.546576 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:32.547294 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:32.546910 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:32.547294 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:32.547005 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:33.545503 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:33.545360 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:33.545503 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:33.545484 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:34.545067 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:34.545036 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:34.545523 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:34.545043 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:34.545523 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:34.545182 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:34.545523 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:34.545271 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:35.545020 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:35.544992 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:35.545224 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:35.545097 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:36.159017 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:36.158955 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:36.159218 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.159105 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:36.159218 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.159175 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret podName:125f4810-5dbf-4e8f-9eb0-ecd8e22564d4 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:44.159153757 +0000 UTC m=+18.068668082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret") pod "global-pull-secret-syncer-mjbdx" (UID: "125f4810-5dbf-4e8f-9eb0-ecd8e22564d4") : object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:36.260328 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:36.260292 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:36.260504 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.260487 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:36.260583 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.260554 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:44.260533828 +0000 UTC m=+18.170048156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:36.361088 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:36.360814 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:36.361088 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.361027 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 02:59:36.361088 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.361052 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 02:59:36.361088 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.361064 2564 projected.go:196] Error preparing data for projected volume kube-api-access-kszw6 for pod openshift-network-diagnostics/network-check-target-dntqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:36.361444 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.361129 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6 podName:8df8eb28-db42-4b67-845f-604b94c50370 nodeName:}" failed. No retries permitted until 2026-06-09 02:59:44.361109667 +0000 UTC m=+18.270623996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kszw6" (UniqueName: "kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6") pod "network-check-target-dntqr" (UID: "8df8eb28-db42-4b67-845f-604b94c50370") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:36.544396 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:36.544316 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:36.544549 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.544463 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:36.544549 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:36.544513 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:36.544664 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:36.544565 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:37.544426 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:37.544396 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:37.544879 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:37.544532 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:38.548734 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:38.548702 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:38.549190 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:38.548717 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:38.549190 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:38.548810 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:38.549190 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:38.548928 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:39.545057 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:39.545030 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:39.545282 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:39.545151 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:40.544665 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:40.544605 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:40.545109 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:40.544740 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:40.545109 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:40.544785 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:40.545109 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:40.544892 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:41.545364 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:41.545290 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:41.545781 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:41.545425 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:42.545143 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:42.545106 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:42.545325 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:42.545234 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:42.545325 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:42.545298 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:42.545436 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:42.545414 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:43.545319 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:43.545298 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:43.545424 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:43.545397 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:44.220721 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:44.220668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:44.221241 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.220857 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:44.221241 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.220939 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret podName:125f4810-5dbf-4e8f-9eb0-ecd8e22564d4 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:00.220918591 +0000 UTC m=+34.130432917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret") pod "global-pull-secret-syncer-mjbdx" (UID: "125f4810-5dbf-4e8f-9eb0-ecd8e22564d4") : object "kube-system"/"original-pull-secret" not registered Jun 09 02:59:44.321757 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:44.321722 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:44.321926 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.321865 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:44.321926 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.321922 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:00.321904702 +0000 UTC m=+34.231419026 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 02:59:44.422077 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:44.422037 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:44.422264 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.422184 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 02:59:44.422264 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.422220 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 02:59:44.422264 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.422234 2564 projected.go:196] Error preparing data for projected volume kube-api-access-kszw6 for pod openshift-network-diagnostics/network-check-target-dntqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:44.422431 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.422293 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6 podName:8df8eb28-db42-4b67-845f-604b94c50370 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:00.422276521 +0000 UTC m=+34.331790849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kszw6" (UniqueName: "kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6") pod "network-check-target-dntqr" (UID: "8df8eb28-db42-4b67-845f-604b94c50370") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 02:59:44.545129 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:44.545046 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:44.545290 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.545165 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:44.545290 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:44.545245 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:44.545396 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:44.545367 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:45.544725 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:45.544689 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:45.545171 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:45.544824 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:46.545070 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:46.545039 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:46.545513 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:46.545144 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:46.545513 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:46.545474 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:46.545589 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:46.545565 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:47.545307 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:47.545268 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:47.545811 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:47.545416 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:48.545431 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.545095 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:48.546248 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.545105 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:48.546248 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:48.545530 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:48.546248 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:48.545574 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:48.626986 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.626942 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vww9x" event={"ID":"e2fa6373-a391-4405-8a41-fb03ae52a042","Type":"ContainerStarted","Data":"c11a62eca098f06e9b9012e2e42fb0fa02d5c12bb73121c0caf1494bbc618dcc"} Jun 09 02:59:48.628693 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.628647 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" event={"ID":"6c4646af-5134-4c7f-af61-b304e3a02a71","Type":"ContainerStarted","Data":"94aaa97642e4e1018bc1ae3351e921830081831df0cdf16de4a8401c1436fec5"} Jun 09 02:59:48.630649 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.630555 2564 generic.go:356] "Generic (PLEG): container finished" podID="348a74fc-1fea-4472-ae4a-d26df51b1993" containerID="3b46cc95db66489a04679c662ed26c1165cdf5584b185f839bb89afabff0953a" exitCode=0 Jun 09 02:59:48.630649 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.630587 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-789lw" event={"ID":"348a74fc-1fea-4472-ae4a-d26df51b1993","Type":"ContainerDied","Data":"3b46cc95db66489a04679c662ed26c1165cdf5584b185f839bb89afabff0953a"} Jun 09 02:59:48.632795 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.632770 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r2p87" event={"ID":"1dc94e9a-d865-4694-9a6a-41f628b714de","Type":"ContainerStarted","Data":"3a7ac01e92d43500f8381bd7d045949e6d1bf009483975202f4fe5befbe7cf86"} Jun 09 02:59:48.634501 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.634479 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dg64b" event={"ID":"b73b4444-c489-4e85-aabd-b69d0c1553b3","Type":"ContainerStarted","Data":"b817887e701d04f441bf042f579e03f1183b670e70e35db5493a5b3ed906815f"} Jun 09 02:59:48.637037 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.637008 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"332bb3da43306cc10b1e2215759749a2d8c0f1d8f91507905c3ceeab7a747ade"} Jun 09 02:59:48.637106 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.637085 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"5ec30dd77de80b1faf76c172d1bde4546dfb435c7b3ac3d34601014b83cf658b"} Jun 09 02:59:48.637154 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.637136 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"e4d6dac290bf9b08f686d8c060aeab4ce0f02f9f0a5fea8e230b1c15d6f2c85c"} Jun 09 02:59:48.637224 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.637184 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"37f9adcea97c7d289ad34c02869e0579f65e495968c29bee461c9c7165cebc76"} Jun 09 02:59:48.638792 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.638752 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vww9x" podStartSLOduration=4.003046163 podStartE2EDuration="22.638739695s" totalImagesPullingTime="18.635693532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:48.638299112 +0000 UTC m=+22.547813458" watchObservedRunningTime="2026-06-09 02:59:48.638739695 +0000 UTC m=+22.548254042" Jun 09 02:59:48.638940 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.638897 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" event={"ID":"ac0436e3-9fbb-4ed4-a153-1fda49d2d527","Type":"ContainerStarted","Data":"c8014547f741d88354a4a901c4c2f5eeaa1ae5548914e85a9947f281294a570c"} Jun 09 02:59:48.640691 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.640669 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xhqhk" event={"ID":"a2adb943-7cd5-43df-bdce-3309518d60c5","Type":"ContainerStarted","Data":"97c6c0f7336b4ab6c1ac0cc65308432f5246598135cbc5a4671839a472b0a1de"} Jun 09 02:59:48.651400 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.651349 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r2p87" podStartSLOduration=3.960107076 podStartE2EDuration="22.65133726s" totalImagesPullingTime="18.691230184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:48.651152252 +0000 UTC m=+22.560666600" watchObservedRunningTime="2026-06-09 02:59:48.65133726 +0000 UTC m=+22.560851607" Jun 09 02:59:48.681579 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.681538 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dg64b" podStartSLOduration=4.028263943 podStartE2EDuration="22.681521363s" totalImagesPullingTime="18.65325742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:48.66796123 +0000 UTC m=+22.577475576" watchObservedRunningTime="2026-06-09 02:59:48.681521363 +0000 UTC m=+22.591035711" Jun 09 02:59:48.699091 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:48.699052 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rfdn2" podStartSLOduration=4.063190747 podStartE2EDuration="22.699039647s" totalImagesPullingTime="18.6358489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:48.680876354 +0000 UTC m=+22.590390721" watchObservedRunningTime="2026-06-09 02:59:48.699039647 +0000 UTC m=+22.608553995" Jun 09 02:59:49.200595 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.200569 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 09 02:59:49.545369 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.545312 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:49.545600 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:49.545413 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:49.552998 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.552919 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-09T02:59:49.200590895Z","UUID":"ba0b202f-83b4-4ed0-a3e9-23021251638e","Handler":null,"Name":"","Endpoint":""} Jun 09 02:59:49.554651 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.554630 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 09 02:59:49.554769 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.554674 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 09 02:59:49.646589 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.646556 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"e895c920d0448c73e4d9f7629b4bbd793892ee417a30614a3616e5a2f3274c78"} Jun 09 02:59:49.646749 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.646661 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"54b076d3ae243fdd8c05cb4b4d68d0ba9b08a94a7b7921e2458be5a97d85fcad"} Jun 09 02:59:49.648489 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.648462 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" event={"ID":"ac0436e3-9fbb-4ed4-a153-1fda49d2d527","Type":"ContainerStarted","Data":"ac57dc600efb45df9ccb2167722442d35a2fca1d08896584230e1044d44968a7"} Jun 09 02:59:49.649994 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.649928 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wp5c7" event={"ID":"0a15392b-3ee8-4387-a5c4-83617bea6bda","Type":"ContainerStarted","Data":"ca3b0e8052176d54d17476316f54bb99df41947b9c7b547144c80a406d9c026b"} Jun 09 02:59:49.662587 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.662542 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wp5c7" podStartSLOduration=5.011642038 podStartE2EDuration="23.662531302s" totalImagesPullingTime="18.650889264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:49.662150552 +0000 UTC m=+23.571664898" watchObservedRunningTime="2026-06-09 02:59:49.662531302 +0000 UTC m=+23.572045649" Jun 09 02:59:49.662708 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:49.662641 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xhqhk" podStartSLOduration=5.021904069 podStartE2EDuration="23.662635324s" totalImagesPullingTime="18.640731255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:48.712090607 +0000 UTC m=+22.621604954" watchObservedRunningTime="2026-06-09 02:59:49.662635324 +0000 UTC m=+23.572149675" Jun 09 02:59:50.545030 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:50.544966 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:50.545184 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:50.545088 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:50.545184 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:50.545124 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:50.545339 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:50.545212 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:50.654145 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:50.654078 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" event={"ID":"ac0436e3-9fbb-4ed4-a153-1fda49d2d527","Type":"ContainerStarted","Data":"a91d635f810b5b3d2fe94c330da3e6c41c9fb9025aff82b8186926b38d079053"} Jun 09 02:59:50.669364 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:50.669320 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqcfn" podStartSLOduration=4.016847315 podStartE2EDuration="24.669306705s" totalImagesPullingTime="20.65245939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:50.66887182 +0000 UTC m=+24.578386168" watchObservedRunningTime="2026-06-09 02:59:50.669306705 +0000 UTC m=+24.578821052" Jun 09 02:59:51.158271 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:51.158238 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:51.158964 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:51.158938 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:51.544983 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:51.544948 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:51.545148 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:51.545070 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:51.659703 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:51.659667 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"ef0a6fa78e2212176fb6b7b2c578a7d24f20c42c2a8c064d1e89a2242576adfa"} Jun 09 02:59:52.545331 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:52.545137 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:52.545511 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:52.545150 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:52.545511 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:52.545446 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:52.545627 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:52.545507 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:53.544332 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:53.544306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:53.544988 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:53.544405 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:53.665844 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:53.665814 2564 generic.go:356] "Generic (PLEG): container finished" podID="348a74fc-1fea-4472-ae4a-d26df51b1993" containerID="856cf6d804fa110d2230c6e5f4bb005d0510af540e216b75e31897cf723a85a4" exitCode=0 Jun 09 02:59:53.665993 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:53.665897 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-789lw" event={"ID":"348a74fc-1fea-4472-ae4a-d26df51b1993","Type":"ContainerDied","Data":"856cf6d804fa110d2230c6e5f4bb005d0510af540e216b75e31897cf723a85a4"} Jun 09 02:59:53.669680 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:53.669659 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" event={"ID":"db8ebec1-7194-4799-a74b-bc212211f99e","Type":"ContainerStarted","Data":"d025069b5291976e2de60c17c349582791e0d6e7f1efca19832569a7be2d2975"} Jun 09 02:59:53.669967 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:53.669945 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:53.685067 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:53.685051 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:53.706515 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:53.706477 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" podStartSLOduration=8.973803304 podStartE2EDuration="27.706466334s" totalImagesPullingTime="18.73266303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 02:59:53.705940667 +0000 UTC m=+27.615455015" watchObservedRunningTime="2026-06-09 02:59:53.706466334 +0000 UTC m=+27.615980680" Jun 09 02:59:54.545134 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.544946 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:54.545577 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.545010 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:54.545577 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:54.545219 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:54.545577 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:54.545319 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:54.674287 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.674189 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:54.674444 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.674296 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:54.691753 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.691731 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 02:59:54.932884 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.932855 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dntqr"] Jun 09 02:59:54.933037 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.932946 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:54.933037 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:54.933029 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:54.935940 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.935914 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mjbdx"] Jun 09 02:59:54.936065 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.936008 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:54.936155 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:54.936130 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:54.936488 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.936467 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l65ml"] Jun 09 02:59:54.936590 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:54.936578 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:54.936701 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:54.936684 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:55.678214 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:55.678166 2564 generic.go:356] "Generic (PLEG): container finished" podID="348a74fc-1fea-4472-ae4a-d26df51b1993" containerID="57de4380888cfec0e6e45800f9e9ea1fba5066580ba259c631f582bde0a83100" exitCode=0 Jun 09 02:59:55.678656 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:55.678254 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-789lw" event={"ID":"348a74fc-1fea-4472-ae4a-d26df51b1993","Type":"ContainerDied","Data":"57de4380888cfec0e6e45800f9e9ea1fba5066580ba259c631f582bde0a83100"} Jun 09 02:59:56.545467 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:56.545439 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:56.545656 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:56.545546 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:56.545656 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:56.545549 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:56.545774 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:56.545666 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:56.545774 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:56.545700 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:56.545882 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:56.545771 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:57.685187 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:57.685014 2564 generic.go:356] "Generic (PLEG): container finished" podID="348a74fc-1fea-4472-ae4a-d26df51b1993" containerID="30e4a0159f093be913baa5f31dd4b1da72869eb34f7d9ec02f448cc8761a8099" exitCode=0 Jun 09 02:59:57.685518 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:57.685092 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-789lw" event={"ID":"348a74fc-1fea-4472-ae4a-d26df51b1993","Type":"ContainerDied","Data":"30e4a0159f093be913baa5f31dd4b1da72869eb34f7d9ec02f448cc8761a8099"} Jun 09 02:59:58.545123 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:58.545089 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 02:59:58.545381 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:58.545090 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 02:59:58.545381 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:58.545232 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 02:59:58.545381 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:58.545105 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 02:59:58.545381 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:58.545282 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 02:59:58.545685 ip-10-0-130-143 kubenswrapper[2564]: E0609 02:59:58.545384 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 02:59:58.626404 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:58.626379 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xhqhk" Jun 09 02:59:58.626559 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:58.626513 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 09 02:59:58.627114 ip-10-0-130-143 kubenswrapper[2564]: I0609 02:59:58.627093 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xhqhk" Jun 09 03:00:00.249893 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.249854 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 03:00:00.250325 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.250011 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 03:00:00.250325 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.250091 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret podName:125f4810-5dbf-4e8f-9eb0-ecd8e22564d4 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:32.250069357 +0000 UTC m=+66.159583683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret") pod "global-pull-secret-syncer-mjbdx" (UID: "125f4810-5dbf-4e8f-9eb0-ecd8e22564d4") : object "kube-system"/"original-pull-secret" not registered Jun 09 03:00:00.350725 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.350693 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:00:00.350902 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.350845 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 03:00:00.350966 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.350919 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:32.350899916 +0000 UTC m=+66.260414245 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 03:00:00.451873 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.451835 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 03:00:00.452044 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.452008 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 03:00:00.452044 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.452031 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 03:00:00.452044 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.452042 2564 projected.go:196] Error preparing data for projected volume kube-api-access-kszw6 for pod openshift-network-diagnostics/network-check-target-dntqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 03:00:00.452225 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.452095 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6 podName:8df8eb28-db42-4b67-845f-604b94c50370 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:32.452078309 +0000 UTC m=+66.361592644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kszw6" (UniqueName: "kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6") pod "network-check-target-dntqr" (UID: "8df8eb28-db42-4b67-845f-604b94c50370") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 03:00:00.545444 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.545356 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 03:00:00.545444 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.545377 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:00:00.545657 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.545492 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjbdx" podUID="125f4810-5dbf-4e8f-9eb0-ecd8e22564d4" Jun 09 03:00:00.545657 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.545560 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 03:00:00.545657 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.545583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 03:00:00.545817 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:00.545663 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dntqr" podUID="8df8eb28-db42-4b67-845f-604b94c50370" Jun 09 03:00:00.882584 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.882516 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-143.ec2.internal" event="NodeReady" Jun 09 03:00:00.882733 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.882653 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 09 03:00:00.921051 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.921013 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8bgc7"] Jun 09 03:00:00.924177 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.923700 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:00.924573 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.924493 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sc8p8"] Jun 09 03:00:00.925899 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.925879 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 09 03:00:00.926027 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.926008 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-27vjm\"" Jun 09 03:00:00.926111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.925881 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 09 03:00:00.926284 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.926261 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sc8p8" Jun 09 03:00:00.928284 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.928261 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 09 03:00:00.928406 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.928301 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 09 03:00:00.928467 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.928448 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kxf98\"" Jun 09 03:00:00.928536 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.928521 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 09 03:00:00.932512 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.932460 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8bgc7"] Jun 09 03:00:00.934994 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.934976 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sc8p8"] Jun 09 03:00:00.957755 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.957732 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:00.957872 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.957793 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4hz9\" (UniqueName: \"kubernetes.io/projected/acd95e7e-686a-4ba0-990f-38a36a91ba27-kube-api-access-f4hz9\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:00.957929 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.957863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acd95e7e-686a-4ba0-990f-38a36a91ba27-config-volume\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:00.957972 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:00.957924 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acd95e7e-686a-4ba0-990f-38a36a91ba27-tmp-dir\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.058365 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.058325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.058533 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.058377 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxhdq\" (UniqueName: \"kubernetes.io/projected/92273549-70ad-4aab-a95e-7b361eed8781-kube-api-access-bxhdq\") pod \"ingress-canary-sc8p8\" (UID: \"92273549-70ad-4aab-a95e-7b361eed8781\") " pod="openshift-ingress-canary/ingress-canary-sc8p8" Jun 09 03:00:01.058533 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.058409 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4hz9\" (UniqueName: \"kubernetes.io/projected/acd95e7e-686a-4ba0-990f-38a36a91ba27-kube-api-access-f4hz9\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.058533 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.058424 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92273549-70ad-4aab-a95e-7b361eed8781-cert\") pod \"ingress-canary-sc8p8\" (UID: \"92273549-70ad-4aab-a95e-7b361eed8781\") " pod="openshift-ingress-canary/ingress-canary-sc8p8" Jun 09 03:00:01.058533 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.058446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acd95e7e-686a-4ba0-990f-38a36a91ba27-config-volume\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.058533 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.058471 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acd95e7e-686a-4ba0-990f-38a36a91ba27-tmp-dir\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.058533 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:01.058490 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:00:01.059014 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:01.058632 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:01.558610378 +0000 UTC m=+35.468124708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:00:01.059014 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.058810 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acd95e7e-686a-4ba0-990f-38a36a91ba27-tmp-dir\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.059124 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.059096 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acd95e7e-686a-4ba0-990f-38a36a91ba27-config-volume\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.070357 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.070326 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4hz9\" (UniqueName: \"kubernetes.io/projected/acd95e7e-686a-4ba0-990f-38a36a91ba27-kube-api-access-f4hz9\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.159521 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.159448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxhdq\" (UniqueName: \"kubernetes.io/projected/92273549-70ad-4aab-a95e-7b361eed8781-kube-api-access-bxhdq\") pod \"ingress-canary-sc8p8\" (UID: \"92273549-70ad-4aab-a95e-7b361eed8781\") " pod="openshift-ingress-canary/ingress-canary-sc8p8" Jun 09 03:00:01.159521 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.159496 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92273549-70ad-4aab-a95e-7b361eed8781-cert\") pod \"ingress-canary-sc8p8\" (UID: \"92273549-70ad-4aab-a95e-7b361eed8781\") " pod="openshift-ingress-canary/ingress-canary-sc8p8" Jun 09 03:00:01.162312 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.162287 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92273549-70ad-4aab-a95e-7b361eed8781-cert\") pod \"ingress-canary-sc8p8\" (UID: \"92273549-70ad-4aab-a95e-7b361eed8781\") " pod="openshift-ingress-canary/ingress-canary-sc8p8" Jun 09 03:00:01.167081 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.167036 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxhdq\" (UniqueName: \"kubernetes.io/projected/92273549-70ad-4aab-a95e-7b361eed8781-kube-api-access-bxhdq\") pod \"ingress-canary-sc8p8\" (UID: \"92273549-70ad-4aab-a95e-7b361eed8781\") " pod="openshift-ingress-canary/ingress-canary-sc8p8" Jun 09 03:00:01.244947 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.244914 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sc8p8" Jun 09 03:00:01.561992 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.561954 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:01.562666 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:01.562129 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:00:01.562666 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:01.562220 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:02.562182038 +0000 UTC m=+36.471696367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:00:01.719894 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:01.719759 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sc8p8"] Jun 09 03:00:01.722721 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:00:01.722697 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92273549_70ad_4aab_a95e_7b361eed8781.slice/crio-fb5ea26302db90810220221bebd491236ee4d29f3513caa467fe32668e24cf57 WatchSource:0}: Error finding container fb5ea26302db90810220221bebd491236ee4d29f3513caa467fe32668e24cf57: Status 404 returned error can't find the container with id fb5ea26302db90810220221bebd491236ee4d29f3513caa467fe32668e24cf57 Jun 09 03:00:02.545409 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.545376 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 03:00:02.545604 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.545376 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 03:00:02.545970 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.545384 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:00:02.548144 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.548122 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 03:00:02.548273 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.548186 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 03:00:02.548964 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.548847 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 03:00:02.548964 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.548923 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jqjfg\"" Jun 09 03:00:02.549124 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.549086 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-stcvs\"" Jun 09 03:00:02.549124 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.549085 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 03:00:02.569036 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.568391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:02.569036 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:02.568549 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:00:02.569036 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:02.568622 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:04.568599175 +0000 UTC m=+38.478113514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:00:02.696341 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.696302 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sc8p8" event={"ID":"92273549-70ad-4aab-a95e-7b361eed8781","Type":"ContainerStarted","Data":"fb5ea26302db90810220221bebd491236ee4d29f3513caa467fe32668e24cf57"} Jun 09 03:00:02.699325 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.699293 2564 generic.go:356] "Generic (PLEG): container finished" podID="348a74fc-1fea-4472-ae4a-d26df51b1993" containerID="894be72e8c665611d40fff1d06ecd3612e7250ccae74a2606b1f6c2d94396277" exitCode=0 Jun 09 03:00:02.699479 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:02.699344 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-789lw" event={"ID":"348a74fc-1fea-4472-ae4a-d26df51b1993","Type":"ContainerDied","Data":"894be72e8c665611d40fff1d06ecd3612e7250ccae74a2606b1f6c2d94396277"} Jun 09 03:00:03.706623 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:03.706586 2564 generic.go:356] "Generic (PLEG): container finished" podID="348a74fc-1fea-4472-ae4a-d26df51b1993" containerID="13781a398271d59699ad52eeb6a078a02be7d9b495d2e15af23070a9a239a799" exitCode=0 Jun 09 03:00:03.707081 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:03.706643 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-789lw" event={"ID":"348a74fc-1fea-4472-ae4a-d26df51b1993","Type":"ContainerDied","Data":"13781a398271d59699ad52eeb6a078a02be7d9b495d2e15af23070a9a239a799"} Jun 09 03:00:04.581436 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:04.581244 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:04.581617 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:04.581383 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:00:04.581617 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:04.581568 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:08.581544843 +0000 UTC m=+42.491059175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:00:04.710155 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:04.710124 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sc8p8" event={"ID":"92273549-70ad-4aab-a95e-7b361eed8781","Type":"ContainerStarted","Data":"638d44fedf114debed1cf08ad8d5feac5c0e9f61e867e6391177ccf1f4611da8"} Jun 09 03:00:04.713419 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:04.713397 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-789lw" event={"ID":"348a74fc-1fea-4472-ae4a-d26df51b1993","Type":"ContainerStarted","Data":"76d2db7c453a35a9710115fe09bc9a615e5878ea04025feaeebdf53187a846a0"} Jun 09 03:00:04.723487 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:04.723444 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sc8p8" podStartSLOduration=2.6276258710000002 podStartE2EDuration="4.723431619s" totalImagesPullingTime="2.095805748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:00:04.722244248 +0000 UTC m=+38.631758595" watchObservedRunningTime="2026-06-09 03:00:04.723431619 +0000 UTC m=+38.632945965" Jun 09 03:00:04.743019 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:04.742981 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-789lw" podStartSLOduration=9.949478007 podStartE2EDuration="38.742967875s" totalImagesPullingTime="28.793489868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:00:04.742238529 +0000 UTC m=+38.651752877" watchObservedRunningTime="2026-06-09 03:00:04.742967875 +0000 UTC m=+38.652482221" Jun 09 03:00:08.610521 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:08.610488 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:08.610901 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:08.610631 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:00:08.610901 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:08.610690 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:16.610675574 +0000 UTC m=+50.520189900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:00:13.292160 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.292104 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk"] Jun 09 03:00:13.302582 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.302558 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c"] Jun 09 03:00:13.302720 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.302687 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" Jun 09 03:00:13.305215 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.305175 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 09 03:00:13.305215 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.305186 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 09 03:00:13.305392 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.305265 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2bkv6\"" Jun 09 03:00:13.305392 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.305175 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 09 03:00:13.305392 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.305181 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 09 03:00:13.313169 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.313148 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk"] Jun 09 03:00:13.313271 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.313176 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c"] Jun 09 03:00:13.313312 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.313273 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.314846 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.314827 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 09 03:00:13.341633 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.341611 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgkwn\" (UniqueName: \"kubernetes.io/projected/4bb5ad29-f52b-4686-ab19-cf93bde1966a-kube-api-access-dgkwn\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.341757 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.341640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr44j\" (UniqueName: \"kubernetes.io/projected/43f43093-12fb-46a6-a458-fa10fa63f642-kube-api-access-vr44j\") pod \"managed-serviceaccount-addon-agent-796f4574cb-6q4sk\" (UID: \"43f43093-12fb-46a6-a458-fa10fa63f642\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" Jun 09 03:00:13.341757 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.341660 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bb5ad29-f52b-4686-ab19-cf93bde1966a-tmp\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.341757 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.341679 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43f43093-12fb-46a6-a458-fa10fa63f642-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-796f4574cb-6q4sk\" (UID: \"43f43093-12fb-46a6-a458-fa10fa63f642\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" Jun 09 03:00:13.341870 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.341765 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4bb5ad29-f52b-4686-ab19-cf93bde1966a-klusterlet-config\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.442437 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.442406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgkwn\" (UniqueName: \"kubernetes.io/projected/4bb5ad29-f52b-4686-ab19-cf93bde1966a-kube-api-access-dgkwn\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.442437 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.442439 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr44j\" (UniqueName: \"kubernetes.io/projected/43f43093-12fb-46a6-a458-fa10fa63f642-kube-api-access-vr44j\") pod \"managed-serviceaccount-addon-agent-796f4574cb-6q4sk\" (UID: \"43f43093-12fb-46a6-a458-fa10fa63f642\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" Jun 09 03:00:13.442687 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.442460 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bb5ad29-f52b-4686-ab19-cf93bde1966a-tmp\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.442687 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.442590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43f43093-12fb-46a6-a458-fa10fa63f642-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-796f4574cb-6q4sk\" (UID: \"43f43093-12fb-46a6-a458-fa10fa63f642\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" Jun 09 03:00:13.442687 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.442639 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4bb5ad29-f52b-4686-ab19-cf93bde1966a-klusterlet-config\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.442838 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.442805 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bb5ad29-f52b-4686-ab19-cf93bde1966a-tmp\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.446549 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.446519 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4bb5ad29-f52b-4686-ab19-cf93bde1966a-klusterlet-config\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.446657 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.446607 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43f43093-12fb-46a6-a458-fa10fa63f642-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-796f4574cb-6q4sk\" (UID: \"43f43093-12fb-46a6-a458-fa10fa63f642\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" Jun 09 03:00:13.449013 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.448986 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgkwn\" (UniqueName: \"kubernetes.io/projected/4bb5ad29-f52b-4686-ab19-cf93bde1966a-kube-api-access-dgkwn\") pod \"klusterlet-addon-workmgr-779654bc88-8lb6c\" (UID: \"4bb5ad29-f52b-4686-ab19-cf93bde1966a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.449095 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.448986 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr44j\" (UniqueName: \"kubernetes.io/projected/43f43093-12fb-46a6-a458-fa10fa63f642-kube-api-access-vr44j\") pod \"managed-serviceaccount-addon-agent-796f4574cb-6q4sk\" (UID: \"43f43093-12fb-46a6-a458-fa10fa63f642\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" Jun 09 03:00:13.624658 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.624579 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" Jun 09 03:00:13.631754 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.631728 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:13.762705 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.762671 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk"] Jun 09 03:00:13.765792 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:00:13.765768 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43f43093_12fb_46a6_a458_fa10fa63f642.slice/crio-b41d352b867a447a5ff760b0dac899a569cb0af8d5d491677d370e712c1703a7 WatchSource:0}: Error finding container b41d352b867a447a5ff760b0dac899a569cb0af8d5d491677d370e712c1703a7: Status 404 returned error can't find the container with id b41d352b867a447a5ff760b0dac899a569cb0af8d5d491677d370e712c1703a7 Jun 09 03:00:13.779400 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:13.779379 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c"] Jun 09 03:00:13.782381 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:00:13.782357 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb5ad29_f52b_4686_ab19_cf93bde1966a.slice/crio-2185a287e5f0ef66632f6fea72b083da72a047bdd40af28942aadd8a6177057c WatchSource:0}: Error finding container 2185a287e5f0ef66632f6fea72b083da72a047bdd40af28942aadd8a6177057c: Status 404 returned error can't find the container with id 2185a287e5f0ef66632f6fea72b083da72a047bdd40af28942aadd8a6177057c Jun 09 03:00:14.738783 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:14.738737 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" event={"ID":"43f43093-12fb-46a6-a458-fa10fa63f642","Type":"ContainerStarted","Data":"b41d352b867a447a5ff760b0dac899a569cb0af8d5d491677d370e712c1703a7"} Jun 09 03:00:14.740098 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:14.740069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" event={"ID":"4bb5ad29-f52b-4686-ab19-cf93bde1966a","Type":"ContainerStarted","Data":"2185a287e5f0ef66632f6fea72b083da72a047bdd40af28942aadd8a6177057c"} Jun 09 03:00:16.667609 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:16.667572 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:16.668077 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:16.667710 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:00:16.668077 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:16.667788 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:00:32.667764725 +0000 UTC m=+66.577279058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:00:19.755538 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:19.755449 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" event={"ID":"43f43093-12fb-46a6-a458-fa10fa63f642","Type":"ContainerStarted","Data":"1c6f39333f8dff980d0b08ded237d4cee898573a59c22692ed604ca70b5777d7"} Jun 09 03:00:19.756744 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:19.756723 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" event={"ID":"4bb5ad29-f52b-4686-ab19-cf93bde1966a","Type":"ContainerStarted","Data":"28f3d13564518b9e87dd90323735f6cfa34f031fb9e3fac3aa3ce4438c62b98d"} Jun 09 03:00:19.756932 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:19.756918 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:19.758406 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:19.758386 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:00:19.767592 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:19.767463 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" podStartSLOduration=1.466813628 podStartE2EDuration="6.767449276s" totalImagesPullingTime="5.300635648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:00:19.767124077 +0000 UTC m=+53.676638425" watchObservedRunningTime="2026-06-09 03:00:19.767449276 +0000 UTC m=+53.676963623" Jun 09 03:00:19.779790 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:19.779745 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" podStartSLOduration=1.079131969 podStartE2EDuration="6.779735664s" totalImagesPullingTime="5.700603695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:00:19.778812243 +0000 UTC m=+53.688326589" watchObservedRunningTime="2026-06-09 03:00:19.779735664 +0000 UTC m=+53.689250025" Jun 09 03:00:26.695549 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:26.695523 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g7wpr" Jun 09 03:00:32.285907 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.285868 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 03:00:32.288594 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.288577 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 03:00:32.298370 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.298350 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/125f4810-5dbf-4e8f-9eb0-ecd8e22564d4-original-pull-secret\") pod \"global-pull-secret-syncer-mjbdx\" (UID: \"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4\") " pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 03:00:32.386745 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.386717 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:00:32.389019 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.389002 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 03:00:32.397055 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:32.397036 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 03:00:32.397123 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:32.397087 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 03:01:36.397073931 +0000 UTC m=+130.306588255 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : secret "metrics-daemon-secret" not found Jun 09 03:00:32.487097 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.487064 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 03:00:32.489228 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.489188 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 03:00:32.500058 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.500041 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 03:00:32.510791 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.510762 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kszw6\" (UniqueName: \"kubernetes.io/projected/8df8eb28-db42-4b67-845f-604b94c50370-kube-api-access-kszw6\") pod \"network-check-target-dntqr\" (UID: \"8df8eb28-db42-4b67-845f-604b94c50370\") " pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 03:00:32.558586 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.558523 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjbdx" Jun 09 03:00:32.567730 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.567608 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-stcvs\"" Jun 09 03:00:32.576272 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.576250 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 03:00:32.687561 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.687531 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mjbdx"] Jun 09 03:00:32.688831 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.688765 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:00:32.688916 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:32.688902 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:00:32.688969 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:00:32.688960 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:01:04.688945187 +0000 UTC m=+98.598459515 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:00:32.690886 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:00:32.690858 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125f4810_5dbf_4e8f_9eb0_ecd8e22564d4.slice/crio-7c4f0f3af0666258deeae4b447bea1e03d3099531c09fb987d3da0890817404a WatchSource:0}: Error finding container 7c4f0f3af0666258deeae4b447bea1e03d3099531c09fb987d3da0890817404a: Status 404 returned error can't find the container with id 7c4f0f3af0666258deeae4b447bea1e03d3099531c09fb987d3da0890817404a Jun 09 03:00:32.704805 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.704785 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dntqr"] Jun 09 03:00:32.707432 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:00:32.707410 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df8eb28_db42_4b67_845f_604b94c50370.slice/crio-0c169249487aa73479ad9d759dff668b27621d8b4512ffb1cc052ad06e590d46 WatchSource:0}: Error finding container 0c169249487aa73479ad9d759dff668b27621d8b4512ffb1cc052ad06e590d46: Status 404 returned error can't find the container with id 0c169249487aa73479ad9d759dff668b27621d8b4512ffb1cc052ad06e590d46 Jun 09 03:00:32.788424 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.788386 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dntqr" event={"ID":"8df8eb28-db42-4b67-845f-604b94c50370","Type":"ContainerStarted","Data":"0c169249487aa73479ad9d759dff668b27621d8b4512ffb1cc052ad06e590d46"} Jun 09 03:00:32.789467 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:32.789445 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mjbdx" event={"ID":"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4","Type":"ContainerStarted","Data":"7c4f0f3af0666258deeae4b447bea1e03d3099531c09fb987d3da0890817404a"} Jun 09 03:00:37.804734 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:37.804695 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dntqr" event={"ID":"8df8eb28-db42-4b67-845f-604b94c50370","Type":"ContainerStarted","Data":"b12679f1448d64195aae3496776ee008fa716e22aaff15239b188420e8c4b102"} Jun 09 03:00:37.805162 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:37.804806 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 03:00:37.806217 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:37.806176 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mjbdx" event={"ID":"125f4810-5dbf-4e8f-9eb0-ecd8e22564d4","Type":"ContainerStarted","Data":"d39cfe6ccd232eb25442c130f11474ae5edaf2aa35024bc2bc8122d238e0a3f7"} Jun 09 03:00:37.818391 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:37.818353 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dntqr" podStartSLOduration=67.429865936 podStartE2EDuration="1m11.818340903s" totalImagesPullingTime="4.388474967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:00:37.817070033 +0000 UTC m=+71.726584376" watchObservedRunningTime="2026-06-09 03:00:37.818340903 +0000 UTC m=+71.727855269" Jun 09 03:00:37.829445 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:00:37.829408 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mjbdx" podStartSLOduration=67.420937799 podStartE2EDuration="1m11.829395756s" totalImagesPullingTime="4.408457957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:00:37.829038378 +0000 UTC m=+71.738552736" watchObservedRunningTime="2026-06-09 03:00:37.829395756 +0000 UTC m=+71.738910103" Jun 09 03:01:04.705284 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:01:04.705247 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:01:04.705722 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:01:04.705390 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:01:04.705722 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:01:04.705449 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:08.705433536 +0000 UTC m=+162.614947861 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:01:08.810192 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:01:08.810161 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dntqr" Jun 09 03:01:36.397829 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:01:36.397783 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:01:36.398317 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:01:36.397903 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 03:01:36.398317 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:01:36.397975 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs podName:9df3f892-882c-409c-a7ac-44e81fe9c003 nodeName:}" failed. No retries permitted until 2026-06-09 03:03:38.397961167 +0000 UTC m=+252.307475493 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs") pod "network-metrics-daemon-l65ml" (UID: "9df3f892-882c-409c-a7ac-44e81fe9c003") : secret "metrics-daemon-secret" not found Jun 09 03:02:03.938096 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:03.938061 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8bgc7" podUID="acd95e7e-686a-4ba0-990f-38a36a91ba27" Jun 09 03:02:04.005288 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:04.005260 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8bgc7" Jun 09 03:02:05.571537 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:05.571477 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l65ml" podUID="9df3f892-882c-409c-a7ac-44e81fe9c003" Jun 09 03:02:08.802224 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:08.802168 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:02:08.802672 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:08.802305 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 03:02:08.802672 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:08.802367 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls podName:acd95e7e-686a-4ba0-990f-38a36a91ba27 nodeName:}" failed. No retries permitted until 2026-06-09 03:04:10.802352078 +0000 UTC m=+284.711866404 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls") pod "dns-default-8bgc7" (UID: "acd95e7e-686a-4ba0-990f-38a36a91ba27") : secret "dns-default-metrics-tls" not found Jun 09 03:02:19.544947 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:19.544856 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:02:19.757719 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:19.757678 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" podUID="4bb5ad29-f52b-4686-ab19-cf93bde1966a" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused" Jun 09 03:02:20.042314 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:20.042286 2564 generic.go:356] "Generic (PLEG): container finished" podID="4bb5ad29-f52b-4686-ab19-cf93bde1966a" containerID="28f3d13564518b9e87dd90323735f6cfa34f031fb9e3fac3aa3ce4438c62b98d" exitCode=1 Jun 09 03:02:20.042467 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:20.042363 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" event={"ID":"4bb5ad29-f52b-4686-ab19-cf93bde1966a","Type":"ContainerDied","Data":"28f3d13564518b9e87dd90323735f6cfa34f031fb9e3fac3aa3ce4438c62b98d"} Jun 09 03:02:20.042694 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:20.042678 2564 scope.go:117] "RemoveContainer" containerID="28f3d13564518b9e87dd90323735f6cfa34f031fb9e3fac3aa3ce4438c62b98d" Jun 09 03:02:20.043970 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:20.043917 2564 generic.go:356] "Generic (PLEG): container finished" podID="43f43093-12fb-46a6-a458-fa10fa63f642" containerID="1c6f39333f8dff980d0b08ded237d4cee898573a59c22692ed604ca70b5777d7" exitCode=255 Jun 09 03:02:20.043970 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:20.043956 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" event={"ID":"43f43093-12fb-46a6-a458-fa10fa63f642","Type":"ContainerDied","Data":"1c6f39333f8dff980d0b08ded237d4cee898573a59c22692ed604ca70b5777d7"} Jun 09 03:02:20.049681 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:20.049664 2564 scope.go:117] "RemoveContainer" containerID="1c6f39333f8dff980d0b08ded237d4cee898573a59c22692ed604ca70b5777d7" Jun 09 03:02:21.049812 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:21.049769 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" event={"ID":"4bb5ad29-f52b-4686-ab19-cf93bde1966a","Type":"ContainerStarted","Data":"4c69b3508ab0925f66fdc45b79232dd5616c51b25d344aedf28e088b5629dfff"} Jun 09 03:02:21.050277 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:21.050073 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:02:21.050724 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:21.050704 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-779654bc88-8lb6c" Jun 09 03:02:21.051558 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:21.051536 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796f4574cb-6q4sk" event={"ID":"43f43093-12fb-46a6-a458-fa10fa63f642","Type":"ContainerStarted","Data":"45cafd59e7de07f421a619b7c75795e99202b2ca44986588f374e28589f53eaa"} Jun 09 03:02:24.171740 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.171711 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74b85654cf-49rnq"] Jun 09 03:02:24.173468 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.173453 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.175504 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.175450 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 09 03:02:24.175504 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.175493 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 09 03:02:24.175692 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.175566 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 09 03:02:24.175692 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.175567 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 09 03:02:24.175888 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.175874 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xd9cw\"" Jun 09 03:02:24.175934 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.175876 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 09 03:02:24.176910 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.176891 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 09 03:02:24.187190 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.187170 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74b85654cf-49rnq"] Jun 09 03:02:24.199671 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.199649 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-default-certificate\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.199767 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.199712 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-stats-auth\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.199767 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.199730 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trcfl\" (UniqueName: \"kubernetes.io/projected/d030d15e-f898-4646-ba0b-f21734c72a89-kube-api-access-trcfl\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.199767 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.199746 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.199767 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.199762 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.300904 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.300879 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.300904 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.300906 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.301108 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.300928 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-default-certificate\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.301108 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.300975 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-stats-auth\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.301108 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.300992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trcfl\" (UniqueName: \"kubernetes.io/projected/d030d15e-f898-4646-ba0b-f21734c72a89-kube-api-access-trcfl\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.301108 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:24.301034 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 03:02:24.301108 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:24.301071 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:24.801048177 +0000 UTC m=+178.710562527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : configmap references non-existent config key: service-ca.crt Jun 09 03:02:24.301108 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:24.301090 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:24.801081013 +0000 UTC m=+178.710595337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : secret "router-metrics-certs-default" not found Jun 09 03:02:24.303301 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.303272 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-default-certificate\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.303388 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.303353 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-stats-auth\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.308398 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.308381 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trcfl\" (UniqueName: \"kubernetes.io/projected/d030d15e-f898-4646-ba0b-f21734c72a89-kube-api-access-trcfl\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.803897 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.803868 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.803897 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:24.803898 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:24.804099 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:24.804009 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 03:02:24.804099 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:24.804049 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:25.80403142 +0000 UTC m=+179.713545744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : configmap references non-existent config key: service-ca.crt Jun 09 03:02:24.804099 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:24.804067 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:25.804059856 +0000 UTC m=+179.713574180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : secret "router-metrics-certs-default" not found Jun 09 03:02:25.810966 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:25.810925 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:25.810966 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:25.810967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:25.811390 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:25.811079 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 03:02:25.811390 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:25.811129 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:27.811114677 +0000 UTC m=+181.720629028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : configmap references non-existent config key: service-ca.crt Jun 09 03:02:25.811390 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:25.811145 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:27.811139328 +0000 UTC m=+181.720653652 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : secret "router-metrics-certs-default" not found Jun 09 03:02:27.823770 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:27.823737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:27.823770 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:27.823772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:27.824172 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:27.823880 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 03:02:27.824172 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:27.823893 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:31.823880177 +0000 UTC m=+185.733394502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : configmap references non-existent config key: service-ca.crt Jun 09 03:02:27.824172 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:27.823937 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:31.823924729 +0000 UTC m=+185.733439054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : secret "router-metrics-certs-default" not found Jun 09 03:02:31.529125 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:31.529098 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dg64b_b73b4444-c489-4e85-aabd-b69d0c1553b3/dns-node-resolver/0.log" Jun 09 03:02:31.852613 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:31.852527 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:31.852613 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:31.852566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:31.852818 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:31.852681 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 03:02:31.852818 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:31.852696 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:39.852682601 +0000 UTC m=+193.762196925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : configmap references non-existent config key: service-ca.crt Jun 09 03:02:31.852818 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:31.852738 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:39.852721546 +0000 UTC m=+193.762235887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : secret "router-metrics-certs-default" not found Jun 09 03:02:32.495334 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.495305 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-j4jjz"] Jun 09 03:02:32.497129 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.497114 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.501620 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.501597 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 09 03:02:32.501755 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.501679 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 09 03:02:32.501851 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.501830 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 09 03:02:32.501983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.501895 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 09 03:02:32.502046 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.502009 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wpqgf\"" Jun 09 03:02:32.508415 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.508391 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-j4jjz"] Jun 09 03:02:32.556078 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.556050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-signing-key\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.556416 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.556092 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-signing-cabundle\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.556416 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.556115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z65m\" (UniqueName: \"kubernetes.io/projected/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-kube-api-access-4z65m\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.656632 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.656591 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-signing-key\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.656749 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.656655 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-signing-cabundle\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.656749 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.656687 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z65m\" (UniqueName: \"kubernetes.io/projected/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-kube-api-access-4z65m\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.657267 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.657250 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-signing-cabundle\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.658952 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.658931 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-signing-key\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.663743 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.663724 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z65m\" (UniqueName: \"kubernetes.io/projected/87ff605f-3abb-4cc2-a7bb-3d98e73f159a-kube-api-access-4z65m\") pod \"service-ca-7d74d578b8-j4jjz\" (UID: \"87ff605f-3abb-4cc2-a7bb-3d98e73f159a\") " pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.809924 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.809856 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" Jun 09 03:02:32.922979 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.922951 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-j4jjz"] Jun 09 03:02:32.925390 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:02:32.925356 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ff605f_3abb_4cc2_a7bb_3d98e73f159a.slice/crio-2136254631a0ec016f75ffaf5e7f186f6b2ee2841c7b538e49f95439730ee1ba WatchSource:0}: Error finding container 2136254631a0ec016f75ffaf5e7f186f6b2ee2841c7b538e49f95439730ee1ba: Status 404 returned error can't find the container with id 2136254631a0ec016f75ffaf5e7f186f6b2ee2841c7b538e49f95439730ee1ba Jun 09 03:02:32.931307 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:32.931290 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vww9x_e2fa6373-a391-4405-8a41-fb03ae52a042/node-ca/0.log" Jun 09 03:02:33.083962 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:33.083900 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" event={"ID":"87ff605f-3abb-4cc2-a7bb-3d98e73f159a","Type":"ContainerStarted","Data":"2136254631a0ec016f75ffaf5e7f186f6b2ee2841c7b538e49f95439730ee1ba"} Jun 09 03:02:33.529853 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:33.529827 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sc8p8_92273549-70ad-4aab-a95e-7b361eed8781/serve-healthcheck-canary/0.log" Jun 09 03:02:36.094528 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:36.094491 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" event={"ID":"87ff605f-3abb-4cc2-a7bb-3d98e73f159a","Type":"ContainerStarted","Data":"fb70a74cd41c0d2ebe8341326e2737b1cedd987496a9869330d52d246e9dd49e"} Jun 09 03:02:36.108135 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:36.107912 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-j4jjz" podStartSLOduration=1.798458852 podStartE2EDuration="4.107891482s" totalImagesPullingTime="2.30943263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:02:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:02:36.107107176 +0000 UTC m=+190.016621522" watchObservedRunningTime="2026-06-09 03:02:36.107891482 +0000 UTC m=+190.017405829" Jun 09 03:02:39.907001 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:39.906968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:39.907001 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:39.907006 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:39.907420 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:39.907113 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 03:02:39.907420 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:39.907190 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:55.907168041 +0000 UTC m=+209.816682365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : secret "router-metrics-certs-default" not found Jun 09 03:02:39.907420 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:39.907217 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle podName:d030d15e-f898-4646-ba0b-f21734c72a89 nodeName:}" failed. No retries permitted until 2026-06-09 03:02:55.907211023 +0000 UTC m=+209.816725347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle") pod "router-default-74b85654cf-49rnq" (UID: "d030d15e-f898-4646-ba0b-f21734c72a89") : configmap references non-existent config key: service-ca.crt Jun 09 03:02:53.510784 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.510752 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xcnf2"] Jun 09 03:02:53.513157 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.513128 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.516218 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.516181 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 09 03:02:53.516357 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.516215 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k87k5\"" Jun 09 03:02:53.516357 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.516220 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 09 03:02:53.516357 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.516250 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 09 03:02:53.516357 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.516288 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 09 03:02:53.522996 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.522975 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xcnf2"] Jun 09 03:02:53.602108 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.602069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/810d93c5-2966-46af-8522-eb4921bf0dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.602108 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.602107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/810d93c5-2966-46af-8522-eb4921bf0dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.602346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.602125 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8bps\" (UniqueName: \"kubernetes.io/projected/810d93c5-2966-46af-8522-eb4921bf0dbe-kube-api-access-t8bps\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.602346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.602181 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/810d93c5-2966-46af-8522-eb4921bf0dbe-data-volume\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.602346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.602271 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/810d93c5-2966-46af-8522-eb4921bf0dbe-crio-socket\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.702792 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.702757 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/810d93c5-2966-46af-8522-eb4921bf0dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.702932 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.702813 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/810d93c5-2966-46af-8522-eb4921bf0dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.702932 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.702842 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8bps\" (UniqueName: \"kubernetes.io/projected/810d93c5-2966-46af-8522-eb4921bf0dbe-kube-api-access-t8bps\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.702932 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.702884 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/810d93c5-2966-46af-8522-eb4921bf0dbe-data-volume\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.702932 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.702915 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/810d93c5-2966-46af-8522-eb4921bf0dbe-crio-socket\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.703115 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.703003 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/810d93c5-2966-46af-8522-eb4921bf0dbe-crio-socket\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.703311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.703291 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/810d93c5-2966-46af-8522-eb4921bf0dbe-data-volume\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.703396 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.703362 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/810d93c5-2966-46af-8522-eb4921bf0dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.705128 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.705106 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/810d93c5-2966-46af-8522-eb4921bf0dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.711532 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.711508 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8bps\" (UniqueName: \"kubernetes.io/projected/810d93c5-2966-46af-8522-eb4921bf0dbe-kube-api-access-t8bps\") pod \"insights-runtime-extractor-xcnf2\" (UID: \"810d93c5-2966-46af-8522-eb4921bf0dbe\") " pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.823347 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.823281 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xcnf2" Jun 09 03:02:53.940119 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:53.940089 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xcnf2"] Jun 09 03:02:53.943710 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:02:53.943687 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod810d93c5_2966_46af_8522_eb4921bf0dbe.slice/crio-33d7ac02d287ee9b84dfb5c20ff84d5b637c60c56105706d85dfed1de12c5394 WatchSource:0}: Error finding container 33d7ac02d287ee9b84dfb5c20ff84d5b637c60c56105706d85dfed1de12c5394: Status 404 returned error can't find the container with id 33d7ac02d287ee9b84dfb5c20ff84d5b637c60c56105706d85dfed1de12c5394 Jun 09 03:02:54.148009 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:54.147926 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xcnf2" event={"ID":"810d93c5-2966-46af-8522-eb4921bf0dbe","Type":"ContainerStarted","Data":"d39e7c196142cc3caeecad6f3aabae0dfed95a2c2b3bdaf2ac19358a95f912d2"} Jun 09 03:02:54.148009 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:54.147977 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xcnf2" event={"ID":"810d93c5-2966-46af-8522-eb4921bf0dbe","Type":"ContainerStarted","Data":"33d7ac02d287ee9b84dfb5c20ff84d5b637c60c56105706d85dfed1de12c5394"} Jun 09 03:02:55.152540 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:55.152508 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xcnf2" event={"ID":"810d93c5-2966-46af-8522-eb4921bf0dbe","Type":"ContainerStarted","Data":"a5021197262271a8bf7d7c477e0718f6636e2b672b4c4c7edb3d84b803b50d61"} Jun 09 03:02:55.918192 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:55.918152 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:55.918404 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:55.918212 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:55.918943 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:55.918909 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d030d15e-f898-4646-ba0b-f21734c72a89-service-ca-bundle\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:55.920870 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:55.920848 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d030d15e-f898-4646-ba0b-f21734c72a89-metrics-certs\") pod \"router-default-74b85654cf-49rnq\" (UID: \"d030d15e-f898-4646-ba0b-f21734c72a89\") " pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:55.983543 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:55.983516 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xd9cw\"" Jun 09 03:02:55.992120 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:55.992091 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:56.437152 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:56.437126 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74b85654cf-49rnq"] Jun 09 03:02:56.440275 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:02:56.440245 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd030d15e_f898_4646_ba0b_f21734c72a89.slice/crio-cef13f25eac951991b574333c1bb36a4eac337d15034fa6ddedc3b7dabfc00d5 WatchSource:0}: Error finding container cef13f25eac951991b574333c1bb36a4eac337d15034fa6ddedc3b7dabfc00d5: Status 404 returned error can't find the container with id cef13f25eac951991b574333c1bb36a4eac337d15034fa6ddedc3b7dabfc00d5 Jun 09 03:02:57.164244 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:57.164192 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xcnf2" event={"ID":"810d93c5-2966-46af-8522-eb4921bf0dbe","Type":"ContainerStarted","Data":"ae0b36ef22b5c9b4c8db931c0cb126b00fa21044b88b5f95084993ba604586dd"} Jun 09 03:02:57.165481 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:57.165456 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74b85654cf-49rnq" event={"ID":"d030d15e-f898-4646-ba0b-f21734c72a89","Type":"ContainerStarted","Data":"424f3b4b324cec98cd3d6f42919d1513ed63787aa2b2d4f22b867f59266e97c9"} Jun 09 03:02:57.165636 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:57.165494 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74b85654cf-49rnq" event={"ID":"d030d15e-f898-4646-ba0b-f21734c72a89","Type":"ContainerStarted","Data":"cef13f25eac951991b574333c1bb36a4eac337d15034fa6ddedc3b7dabfc00d5"} Jun 09 03:02:57.178847 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:57.178800 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xcnf2" podStartSLOduration=1.8714857980000001 podStartE2EDuration="4.178786129s" totalImagesPullingTime="2.307300331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 03:02:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:02:57.178023025 +0000 UTC m=+211.087537369" watchObservedRunningTime="2026-06-09 03:02:57.178786129 +0000 UTC m=+211.088300476" Jun 09 03:02:57.193361 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:57.193321 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74b85654cf-49rnq" podStartSLOduration=33.19330789 podStartE2EDuration="33.19330789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:02:57.192737228 +0000 UTC m=+211.102251574" watchObservedRunningTime="2026-06-09 03:02:57.19330789 +0000 UTC m=+211.102822237" Jun 09 03:02:57.992520 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:57.992485 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:57.995084 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:57.995060 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:58.169055 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:58.169027 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:58.169986 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:58.169963 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74b85654cf-49rnq" Jun 09 03:02:58.926442 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:58.926403 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9"] Jun 09 03:02:58.928679 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:58.928662 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" Jun 09 03:02:58.930482 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:58.930457 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 09 03:02:58.930597 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:58.930538 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-fcmqh\"" Jun 09 03:02:58.937193 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:58.937151 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9"] Jun 09 03:02:59.041013 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:59.040987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/10d21e65-2a1a-4e99-bf92-f3385a1ab81b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-9bwz9\" (UID: \"10d21e65-2a1a-4e99-bf92-f3385a1ab81b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" Jun 09 03:02:59.141363 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:59.141336 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/10d21e65-2a1a-4e99-bf92-f3385a1ab81b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-9bwz9\" (UID: \"10d21e65-2a1a-4e99-bf92-f3385a1ab81b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" Jun 09 03:02:59.141481 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:59.141464 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 09 03:02:59.141549 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:02:59.141540 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d21e65-2a1a-4e99-bf92-f3385a1ab81b-tls-certificates podName:10d21e65-2a1a-4e99-bf92-f3385a1ab81b nodeName:}" failed. No retries permitted until 2026-06-09 03:02:59.641521714 +0000 UTC m=+213.551036043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/10d21e65-2a1a-4e99-bf92-f3385a1ab81b-tls-certificates") pod "prometheus-operator-admission-webhook-5b6b8f594-9bwz9" (UID: "10d21e65-2a1a-4e99-bf92-f3385a1ab81b") : secret "prometheus-operator-admission-webhook-tls" not found Jun 09 03:02:59.647138 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:59.647104 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/10d21e65-2a1a-4e99-bf92-f3385a1ab81b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-9bwz9\" (UID: \"10d21e65-2a1a-4e99-bf92-f3385a1ab81b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" Jun 09 03:02:59.649347 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:59.649320 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/10d21e65-2a1a-4e99-bf92-f3385a1ab81b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-9bwz9\" (UID: \"10d21e65-2a1a-4e99-bf92-f3385a1ab81b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" Jun 09 03:02:59.837834 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:59.837806 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" Jun 09 03:02:59.955958 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:02:59.955930 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9"] Jun 09 03:02:59.958687 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:02:59.958653 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d21e65_2a1a_4e99_bf92_f3385a1ab81b.slice/crio-fcaf3a5d019620fbc1d1f2506466e8ba7531ed95a27bad77fd43187e39cdf615 WatchSource:0}: Error finding container fcaf3a5d019620fbc1d1f2506466e8ba7531ed95a27bad77fd43187e39cdf615: Status 404 returned error can't find the container with id fcaf3a5d019620fbc1d1f2506466e8ba7531ed95a27bad77fd43187e39cdf615 Jun 09 03:03:00.175904 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:00.175834 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" event={"ID":"10d21e65-2a1a-4e99-bf92-f3385a1ab81b","Type":"ContainerStarted","Data":"fcaf3a5d019620fbc1d1f2506466e8ba7531ed95a27bad77fd43187e39cdf615"} Jun 09 03:03:02.182121 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:02.182083 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" event={"ID":"10d21e65-2a1a-4e99-bf92-f3385a1ab81b","Type":"ContainerStarted","Data":"f6980d53d4c54bfba77415aa76670de74f39a9e5332b36f7dfb7ae71218cbf8f"} Jun 09 03:03:02.182502 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:02.182289 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" Jun 09 03:03:02.187192 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:02.187170 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" Jun 09 03:03:02.194830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:02.194793 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-9bwz9" podStartSLOduration=2.984458555 podStartE2EDuration="4.194779657s" totalImagesPullingTime="1.210321102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:02:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:03:02.194518931 +0000 UTC m=+216.104033290" watchObservedRunningTime="2026-06-09 03:03:02.194779657 +0000 UTC m=+216.104294004" Jun 09 03:03:08.330713 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.330679 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k"] Jun 09 03:03:08.335058 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.335040 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.337459 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.337433 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4hnqn\"" Jun 09 03:03:08.337625 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.337568 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 09 03:03:08.337625 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.337586 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 09 03:03:08.337843 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.337827 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 09 03:03:08.337996 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.337980 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 09 03:03:08.338058 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.338039 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 09 03:03:08.338231 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.338216 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 09 03:03:08.361160 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.361134 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k"] Jun 09 03:03:08.362275 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.362251 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6k2ht"] Jun 09 03:03:08.365479 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.365459 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.368107 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.368086 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 09 03:03:08.368232 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.368185 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7k8xb\"" Jun 09 03:03:08.368449 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.368428 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 09 03:03:08.368522 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.368449 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 09 03:03:08.411824 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.411794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.411969 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.411844 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0a7bf72-0ca0-491a-8445-6757ea247410-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.411969 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.411865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.411969 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.411951 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jjgw\" (UniqueName: \"kubernetes.io/projected/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-api-access-7jjgw\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.412101 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.411992 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a7bf72-0ca0-491a-8445-6757ea247410-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.412101 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.412033 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513082 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513044 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a7bf72-0ca0-491a-8445-6757ea247410-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513092 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-metrics-client-ca\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-sys\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513189 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-textfile\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513239 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-tls\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513269 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-accelerators-collector-config\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513668 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513317 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tscqf\" (UniqueName: \"kubernetes.io/projected/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-kube-api-access-tscqf\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513668 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513344 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513668 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513396 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-wtmp\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513668 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0a7bf72-0ca0-491a-8445-6757ea247410-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513668 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513668 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513514 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jjgw\" (UniqueName: \"kubernetes.io/projected/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-api-access-7jjgw\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513668 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513542 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-root\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.513961 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513826 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0a7bf72-0ca0-491a-8445-6757ea247410-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513961 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513885 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a7bf72-0ca0-491a-8445-6757ea247410-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.513961 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.513946 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.515725 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.515704 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.515787 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.515775 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.523215 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.523178 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jjgw\" (UniqueName: \"kubernetes.io/projected/e0a7bf72-0ca0-491a-8445-6757ea247410-kube-api-access-7jjgw\") pod \"kube-state-metrics-57bbf8bfb5-c4b5k\" (UID: \"e0a7bf72-0ca0-491a-8445-6757ea247410\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.613898 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.613824 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-wtmp\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.613898 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.613884 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-root\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614105 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.613911 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-metrics-client-ca\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614105 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.613952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-sys\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614105 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.613972 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-textfile\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614105 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614014 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-wtmp\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614105 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614016 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-root\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614105 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614038 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-sys\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614105 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614067 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-tls\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614105 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614097 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614510 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614122 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-accelerators-collector-config\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614510 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tscqf\" (UniqueName: \"kubernetes.io/projected/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-kube-api-access-tscqf\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614510 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614355 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-textfile\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614678 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614657 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-metrics-client-ca\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.614716 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.614659 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-accelerators-collector-config\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.616384 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.616361 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-tls\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.616472 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.616403 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.621622 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.621602 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tscqf\" (UniqueName: \"kubernetes.io/projected/0a6cfce7-ffb6-4bc3-ba83-d35d2482678c-kube-api-access-tscqf\") pod \"node-exporter-6k2ht\" (UID: \"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c\") " pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.645400 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.645370 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" Jun 09 03:03:08.675072 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.675043 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6k2ht" Jun 09 03:03:08.687437 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:03:08.687408 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6cfce7_ffb6_4bc3_ba83_d35d2482678c.slice/crio-1127b4e0251aea84c837d8b321717d1a8b3193b458bf87466afca30ac46cb680 WatchSource:0}: Error finding container 1127b4e0251aea84c837d8b321717d1a8b3193b458bf87466afca30ac46cb680: Status 404 returned error can't find the container with id 1127b4e0251aea84c837d8b321717d1a8b3193b458bf87466afca30ac46cb680 Jun 09 03:03:08.770590 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:08.770557 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k"] Jun 09 03:03:08.773569 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:03:08.773546 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a7bf72_0ca0_491a_8445_6757ea247410.slice/crio-86813203dc9cae9c64680edf6407845567cacb3c5bb6f090d204eb7410fa188d WatchSource:0}: Error finding container 86813203dc9cae9c64680edf6407845567cacb3c5bb6f090d204eb7410fa188d: Status 404 returned error can't find the container with id 86813203dc9cae9c64680edf6407845567cacb3c5bb6f090d204eb7410fa188d Jun 09 03:03:09.205337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:09.205307 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" event={"ID":"e0a7bf72-0ca0-491a-8445-6757ea247410","Type":"ContainerStarted","Data":"86813203dc9cae9c64680edf6407845567cacb3c5bb6f090d204eb7410fa188d"} Jun 09 03:03:09.206503 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:09.206476 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6k2ht" event={"ID":"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c","Type":"ContainerStarted","Data":"1127b4e0251aea84c837d8b321717d1a8b3193b458bf87466afca30ac46cb680"} Jun 09 03:03:10.212295 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.212253 2564 generic.go:356] "Generic (PLEG): container finished" podID="0a6cfce7-ffb6-4bc3-ba83-d35d2482678c" containerID="1d224cbc7749aed472feb1ce420d77dfb9d37c889370b62e83405b9c866ab232" exitCode=0 Jun 09 03:03:10.212739 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.212336 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6k2ht" event={"ID":"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c","Type":"ContainerDied","Data":"1d224cbc7749aed472feb1ce420d77dfb9d37c889370b62e83405b9c866ab232"} Jun 09 03:03:10.289520 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.289486 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r"] Jun 09 03:03:10.298829 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.298795 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.300816 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.300781 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 09 03:03:10.300952 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.300779 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 09 03:03:10.301104 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.301078 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4grk4\"" Jun 09 03:03:10.301104 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.301100 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 09 03:03:10.301305 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.301134 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 09 03:03:10.301305 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.301147 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 09 03:03:10.301305 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.301287 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6tl67opbj5n95\"" Jun 09 03:03:10.304944 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.304920 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r"] Jun 09 03:03:10.430850 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.430825 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.430951 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.430859 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.430951 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.430885 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-metrics-client-ca\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.430951 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.430928 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.430951 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.430946 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-grpc-tls\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.431077 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.431004 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-tls\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.431077 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.431027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.431077 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.431047 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9znc\" (UniqueName: \"kubernetes.io/projected/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-kube-api-access-p9znc\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.531882 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.531850 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.531998 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.531902 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-grpc-tls\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.531998 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.531964 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-tls\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.532064 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.531995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.532064 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.532022 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9znc\" (UniqueName: \"kubernetes.io/projected/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-kube-api-access-p9znc\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.532140 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.532079 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.532140 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.532112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.532234 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.532157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-metrics-client-ca\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.533357 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.532886 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-metrics-client-ca\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.535540 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.535508 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.536343 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.536111 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-grpc-tls\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.536591 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.536534 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.537151 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.536946 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.537151 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.537001 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.538011 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.537981 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-secret-thanos-querier-tls\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.547066 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.547033 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9znc\" (UniqueName: \"kubernetes.io/projected/65b0ffdc-a288-4086-a2a3-e5c0227ee50b-kube-api-access-p9znc\") pod \"thanos-querier-6dc7d7fff-wlw9r\" (UID: \"65b0ffdc-a288-4086-a2a3-e5c0227ee50b\") " pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.610849 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.610794 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:10.738897 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:10.738823 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r"] Jun 09 03:03:10.741741 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:03:10.741714 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b0ffdc_a288_4086_a2a3_e5c0227ee50b.slice/crio-bf756877bc594190048d36d6914b669fa4f79c8c10bd6ec3f87d2646b60d2a41 WatchSource:0}: Error finding container bf756877bc594190048d36d6914b669fa4f79c8c10bd6ec3f87d2646b60d2a41: Status 404 returned error can't find the container with id bf756877bc594190048d36d6914b669fa4f79c8c10bd6ec3f87d2646b60d2a41 Jun 09 03:03:11.217693 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:11.217657 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6k2ht" event={"ID":"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c","Type":"ContainerStarted","Data":"9281cf1537565ea1caa00dd6d002c5dea423cea4a8ba80aab9cf535d540afd32"} Jun 09 03:03:11.218148 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:11.217722 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6k2ht" event={"ID":"0a6cfce7-ffb6-4bc3-ba83-d35d2482678c","Type":"ContainerStarted","Data":"89e092d0dde564f87f4a76378d34aae6656d1625593d2e2818f5e1a40f9fc226"} Jun 09 03:03:11.218921 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:11.218894 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" event={"ID":"65b0ffdc-a288-4086-a2a3-e5c0227ee50b","Type":"ContainerStarted","Data":"bf756877bc594190048d36d6914b669fa4f79c8c10bd6ec3f87d2646b60d2a41"} Jun 09 03:03:11.221044 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:11.221019 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" event={"ID":"e0a7bf72-0ca0-491a-8445-6757ea247410","Type":"ContainerStarted","Data":"23d9df7d6536186557fe3120463f247aa71c3606cc20b062016da316fade9c00"} Jun 09 03:03:11.221169 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:11.221063 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" event={"ID":"e0a7bf72-0ca0-491a-8445-6757ea247410","Type":"ContainerStarted","Data":"81befb20299fab85d7700c5e99f5e9148566b0d19cc0251f312de9e325d96d04"} Jun 09 03:03:11.221169 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:11.221081 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" event={"ID":"e0a7bf72-0ca0-491a-8445-6757ea247410","Type":"ContainerStarted","Data":"9fb4d66a511222a42391d544a161e13d4c194531676f4c262d6bc7e67bbbaddd"} Jun 09 03:03:11.234230 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:11.234142 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6k2ht" podStartSLOduration=2.384293569 podStartE2EDuration="3.234124504s" totalImagesPullingTime="787.045583ms" totalInitContainerRuntime="62.785352ms" isStatefulPod=true podCreationTimestamp="2026-06-09 03:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:03:11.232115917 +0000 UTC m=+225.141630265" watchObservedRunningTime="2026-06-09 03:03:11.234124504 +0000 UTC m=+225.143638859" Jun 09 03:03:11.247011 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:11.246962 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-c4b5k" podStartSLOduration=1.6437707910000001 podStartE2EDuration="3.246947893s" totalImagesPullingTime="1.603177102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:03:11.24595767 +0000 UTC m=+225.155472045" watchObservedRunningTime="2026-06-09 03:03:11.246947893 +0000 UTC m=+225.156462240" Jun 09 03:03:13.231442 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:13.231411 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" event={"ID":"65b0ffdc-a288-4086-a2a3-e5c0227ee50b","Type":"ContainerStarted","Data":"51434382d12eb6c06bb7d967ff509d664af3c892e4f87e3a3dd32f50a62d960a"} Jun 09 03:03:13.231802 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:13.231482 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" event={"ID":"65b0ffdc-a288-4086-a2a3-e5c0227ee50b","Type":"ContainerStarted","Data":"527cf6d2fa58e8885ca7709e0fb12c41604394229887ce46631d908d6fbebf6e"} Jun 09 03:03:13.231802 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:13.231508 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" event={"ID":"65b0ffdc-a288-4086-a2a3-e5c0227ee50b","Type":"ContainerStarted","Data":"6386c063464bd6a1dda6dbe59982ef6d816cfef7aa1b9b92b53c9ea77da9fdcc"} Jun 09 03:03:14.237476 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.237438 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" event={"ID":"65b0ffdc-a288-4086-a2a3-e5c0227ee50b","Type":"ContainerStarted","Data":"4a86e335d3b3d1ab1787f38e0f7623500dd2b0b2b40c355850b16e2174ebe9b6"} Jun 09 03:03:14.237962 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.237545 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" event={"ID":"65b0ffdc-a288-4086-a2a3-e5c0227ee50b","Type":"ContainerStarted","Data":"7d7cd482a629f944802093fca95d7255cebf5dad735f3e5c90ed53e1f8536d2b"} Jun 09 03:03:14.237962 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.237591 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" event={"ID":"65b0ffdc-a288-4086-a2a3-e5c0227ee50b","Type":"ContainerStarted","Data":"fffa8198759a80a6d1c85d23599852828611193f16c419ae8263dd6b66890a32"} Jun 09 03:03:14.237962 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.237645 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:14.256063 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.256003 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" podStartSLOduration=1.488306611 podStartE2EDuration="4.255984064s" totalImagesPullingTime="2.767677453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:03:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:03:14.255051435 +0000 UTC m=+228.164565781" watchObservedRunningTime="2026-06-09 03:03:14.255984064 +0000 UTC m=+228.165498415" Jun 09 03:03:14.434621 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.434584 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:03:14.439215 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.439177 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.441116 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441090 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 03:03:14.441230 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441212 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ao4hlk6uvpt4g\"" Jun 09 03:03:14.441510 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441489 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 03:03:14.441608 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441544 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 09 03:03:14.441608 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441586 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nnp87\"" Jun 09 03:03:14.441608 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441587 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 03:03:14.441774 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441621 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 03:03:14.441774 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441727 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 03:03:14.441944 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441862 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 03:03:14.441944 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441886 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 03:03:14.441944 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441894 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 03:03:14.441944 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441865 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 03:03:14.441944 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.441902 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 03:03:14.442283 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.442268 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 03:03:14.442349 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.442275 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 03:03:14.442390 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.442288 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 03:03:14.444313 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.444293 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 03:03:14.455356 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.455336 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:03:14.570602 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570503 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570602 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570548 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570602 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570578 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570633 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570653 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570733 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570843 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tdwb\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-kube-api-access-9tdwb\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.570885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570869 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.571296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570896 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.571296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.571296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.571296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.570987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.571296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.571017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.571296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.571046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.571296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.571100 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.571296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.571132 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.671746 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.671710 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.671746 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.671750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.671981 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.671773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.671981 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.671893 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.671981 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.671922 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.671981 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.671944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.671981 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.671977 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672258 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672026 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672258 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672063 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672567 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672511 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672710 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672589 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672710 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672592 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672710 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672644 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672710 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672687 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672913 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672718 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672913 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672743 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672913 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672787 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672913 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672803 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672913 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672828 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672913 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672858 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.672913 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672896 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.673257 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672920 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tdwb\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-kube-api-access-9tdwb\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.673257 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672946 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.673257 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.672972 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.673980 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.673951 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.674725 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.674668 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.676496 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.675907 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.676496 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.676415 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.676496 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.676459 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.677005 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.676978 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.677246 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.677218 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.677514 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.677487 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.677599 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.677537 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.677798 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.677773 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.677929 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.677905 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.678236 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.678194 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.678236 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.678232 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.678637 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.678618 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.678762 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.678740 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.681796 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.681772 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tdwb\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-kube-api-access-9tdwb\") pod \"prometheus-k8s-0\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.751096 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.751061 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:14.880263 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:14.880239 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:03:14.882178 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:03:14.882150 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8659e0d_73c7_4e0b_95a5_0a8901bbf94a.slice/crio-d73143f02faf349eb9ef9b19a723b4408f674a7f77116f5848e368025b09f9cd WatchSource:0}: Error finding container d73143f02faf349eb9ef9b19a723b4408f674a7f77116f5848e368025b09f9cd: Status 404 returned error can't find the container with id d73143f02faf349eb9ef9b19a723b4408f674a7f77116f5848e368025b09f9cd Jun 09 03:03:15.242338 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:15.242293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerStarted","Data":"d73143f02faf349eb9ef9b19a723b4408f674a7f77116f5848e368025b09f9cd"} Jun 09 03:03:16.247484 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:16.247448 2564 generic.go:356] "Generic (PLEG): container finished" podID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerID="df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f" exitCode=0 Jun 09 03:03:16.247484 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:16.247487 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerDied","Data":"df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f"} Jun 09 03:03:20.248217 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:20.248120 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6dc7d7fff-wlw9r" Jun 09 03:03:20.266918 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:20.266887 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerStarted","Data":"ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8"} Jun 09 03:03:20.267051 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:20.266952 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerStarted","Data":"ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61"} Jun 09 03:03:20.267051 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:20.266987 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerStarted","Data":"f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de"} Jun 09 03:03:20.267051 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:20.267013 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerStarted","Data":"2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567"} Jun 09 03:03:20.267051 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:20.267046 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerStarted","Data":"cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6"} Jun 09 03:03:20.267195 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:20.267075 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerStarted","Data":"f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce"} Jun 09 03:03:20.292648 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:20.292588 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.881604528 podStartE2EDuration="6.292568019s" totalImagesPullingTime="4.391126727s" totalInitContainerRuntime="19.836764ms" isStatefulPod=false podCreationTimestamp="2026-06-09 03:03:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:03:20.290639918 +0000 UTC m=+234.200154266" watchObservedRunningTime="2026-06-09 03:03:20.292568019 +0000 UTC m=+234.202082366" Jun 09 03:03:24.751971 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:24.751909 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:03:38.467734 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:38.467696 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:03:38.469969 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:38.469941 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df3f892-882c-409c-a7ac-44e81fe9c003-metrics-certs\") pod \"network-metrics-daemon-l65ml\" (UID: \"9df3f892-882c-409c-a7ac-44e81fe9c003\") " pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:03:38.747616 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:38.747535 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jqjfg\"" Jun 09 03:03:38.756800 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:38.756780 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l65ml" Jun 09 03:03:38.882666 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:38.882641 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l65ml"] Jun 09 03:03:38.885001 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:03:38.884978 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df3f892_882c_409c_a7ac_44e81fe9c003.slice/crio-9ec48c0ddde0f306612cbc2b9fec740dc3fe7bde7579cc8354f326bab75d1cba WatchSource:0}: Error finding container 9ec48c0ddde0f306612cbc2b9fec740dc3fe7bde7579cc8354f326bab75d1cba: Status 404 returned error can't find the container with id 9ec48c0ddde0f306612cbc2b9fec740dc3fe7bde7579cc8354f326bab75d1cba Jun 09 03:03:39.333387 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:39.333344 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l65ml" event={"ID":"9df3f892-882c-409c-a7ac-44e81fe9c003","Type":"ContainerStarted","Data":"9ec48c0ddde0f306612cbc2b9fec740dc3fe7bde7579cc8354f326bab75d1cba"} Jun 09 03:03:40.338657 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:40.338624 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l65ml" event={"ID":"9df3f892-882c-409c-a7ac-44e81fe9c003","Type":"ContainerStarted","Data":"935161be3dfc4375ce798850787e82f41d79597bc7a6838da0e588ca8d213a9a"} Jun 09 03:03:40.339054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:40.338673 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l65ml" event={"ID":"9df3f892-882c-409c-a7ac-44e81fe9c003","Type":"ContainerStarted","Data":"e3396ee423034ffba8144861b96222f652c7859032b592dca4fef48ccd1b5c8b"} Jun 09 03:03:40.352507 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:40.352452 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l65ml" podStartSLOduration=253.540305547 podStartE2EDuration="4m14.352438558s" totalImagesPullingTime="812.133011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 02:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:03:40.351625604 +0000 UTC m=+254.261139951" watchObservedRunningTime="2026-06-09 03:03:40.352438558 +0000 UTC m=+254.261952932" Jun 09 03:03:59.810913 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:03:59.810834 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-c4b5k_e0a7bf72-0ca0-491a-8445-6757ea247410/kube-state-metrics/0.log" Jun 09 03:04:00.009342 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:00.009312 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-c4b5k_e0a7bf72-0ca0-491a-8445-6757ea247410/kube-rbac-proxy-main/0.log" Jun 09 03:04:00.209280 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:00.209258 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-c4b5k_e0a7bf72-0ca0-491a-8445-6757ea247410/kube-rbac-proxy-self/0.log" Jun 09 03:04:00.809359 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:00.809333 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k2ht_0a6cfce7-ffb6-4bc3-ba83-d35d2482678c/init-textfile/0.log" Jun 09 03:04:01.010214 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:01.010177 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k2ht_0a6cfce7-ffb6-4bc3-ba83-d35d2482678c/node-exporter/0.log" Jun 09 03:04:01.209640 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:01.209616 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k2ht_0a6cfce7-ffb6-4bc3-ba83-d35d2482678c/kube-rbac-proxy/0.log" Jun 09 03:04:03.209134 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:03.209109 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8659e0d-73c7-4e0b-95a5-0a8901bbf94a/init-config-reloader/0.log" Jun 09 03:04:03.409589 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:03.409564 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8659e0d-73c7-4e0b-95a5-0a8901bbf94a/prometheus/0.log" Jun 09 03:04:03.609587 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:03.609522 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8659e0d-73c7-4e0b-95a5-0a8901bbf94a/config-reloader/0.log" Jun 09 03:04:03.809454 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:03.809425 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8659e0d-73c7-4e0b-95a5-0a8901bbf94a/thanos-sidecar/0.log" Jun 09 03:04:04.009487 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:04.009461 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8659e0d-73c7-4e0b-95a5-0a8901bbf94a/kube-rbac-proxy-web/0.log" Jun 09 03:04:04.209341 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:04.209314 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8659e0d-73c7-4e0b-95a5-0a8901bbf94a/kube-rbac-proxy/0.log" Jun 09 03:04:04.409046 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:04.408979 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8659e0d-73c7-4e0b-95a5-0a8901bbf94a/kube-rbac-proxy-thanos/0.log" Jun 09 03:04:05.009293 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:05.009264 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-9bwz9_10d21e65-2a1a-4e99-bf92-f3385a1ab81b/prometheus-operator-admission-webhook/0.log" Jun 09 03:04:05.209186 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:05.209159 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/thanos-query/0.log" Jun 09 03:04:05.408496 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:05.408424 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/kube-rbac-proxy-web/0.log" Jun 09 03:04:05.608834 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:05.608814 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/kube-rbac-proxy/0.log" Jun 09 03:04:05.808996 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:05.808977 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/prom-label-proxy/0.log" Jun 09 03:04:06.009070 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:06.009044 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/kube-rbac-proxy-rules/0.log" Jun 09 03:04:06.225422 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:06.225402 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/kube-rbac-proxy-metrics/0.log" Jun 09 03:04:07.006392 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:04:07.006347 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8bgc7" podUID="acd95e7e-686a-4ba0-990f-38a36a91ba27" Jun 09 03:04:07.438847 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:07.438772 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8bgc7" Jun 09 03:04:08.812909 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:08.812884 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dg64b_b73b4444-c489-4e85-aabd-b69d0c1553b3/dns-node-resolver/0.log" Jun 09 03:04:10.210264 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:10.210235 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vww9x_e2fa6373-a391-4405-8a41-fb03ae52a042/node-ca/0.log" Jun 09 03:04:10.412613 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:10.412587 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74b85654cf-49rnq_d030d15e-f898-4646-ba0b-f21734c72a89/router/0.log" Jun 09 03:04:10.813140 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:10.813112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:04:10.814288 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:10.814270 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sc8p8_92273549-70ad-4aab-a95e-7b361eed8781/serve-healthcheck-canary/0.log" Jun 09 03:04:10.815536 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:10.815518 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acd95e7e-686a-4ba0-990f-38a36a91ba27-metrics-tls\") pod \"dns-default-8bgc7\" (UID: \"acd95e7e-686a-4ba0-990f-38a36a91ba27\") " pod="openshift-dns/dns-default-8bgc7" Jun 09 03:04:11.041004 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:11.040977 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-27vjm\"" Jun 09 03:04:11.050359 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:11.050325 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8bgc7" Jun 09 03:04:11.185242 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:11.185169 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8bgc7"] Jun 09 03:04:11.188106 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:04:11.188077 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd95e7e_686a_4ba0_990f_38a36a91ba27.slice/crio-28639bb8efd3cd85061e163beb30ab09f289743b6b53878cb1afb1ca47d0a152 WatchSource:0}: Error finding container 28639bb8efd3cd85061e163beb30ab09f289743b6b53878cb1afb1ca47d0a152: Status 404 returned error can't find the container with id 28639bb8efd3cd85061e163beb30ab09f289743b6b53878cb1afb1ca47d0a152 Jun 09 03:04:11.454854 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:11.454821 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8bgc7" event={"ID":"acd95e7e-686a-4ba0-990f-38a36a91ba27","Type":"ContainerStarted","Data":"28639bb8efd3cd85061e163beb30ab09f289743b6b53878cb1afb1ca47d0a152"} Jun 09 03:04:14.474547 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:14.474486 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8bgc7" event={"ID":"acd95e7e-686a-4ba0-990f-38a36a91ba27","Type":"ContainerStarted","Data":"ad50d3ec5386343e0e88b9358943fc14a94951e27b588bfe14790e38945d9a18"} Jun 09 03:04:14.475184 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:14.475153 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8bgc7" Jun 09 03:04:14.475443 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:14.475423 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8bgc7" event={"ID":"acd95e7e-686a-4ba0-990f-38a36a91ba27","Type":"ContainerStarted","Data":"2cfcdd0a8061ee98387a917191793f604c9c63e17955ebc20e84e8f8f85dd125"} Jun 09 03:04:14.489218 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:14.489156 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8bgc7" podStartSLOduration=251.57928149 podStartE2EDuration="4m14.48913734s" totalImagesPullingTime="2.90985585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:04:14.488705014 +0000 UTC m=+288.398219361" watchObservedRunningTime="2026-06-09 03:04:14.48913734 +0000 UTC m=+288.398651690" Jun 09 03:04:14.752254 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:14.752222 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:14.773276 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:14.773249 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:15.491827 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:15.491803 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:25.482820 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:25.482791 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8bgc7" Jun 09 03:04:26.497675 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:26.497645 2564 kubelet.go:1648] "Image garbage collection succeeded" Jun 09 03:04:32.803030 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:32.802999 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:04:32.805587 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:32.803307 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="prometheus" containerID="cri-o://f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce" gracePeriod=600 Jun 09 03:04:32.805587 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:32.803342 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="config-reloader" containerID="cri-o://cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6" gracePeriod=600 Jun 09 03:04:32.805587 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:32.803373 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8" gracePeriod=600 Jun 09 03:04:32.805587 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:32.803383 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy-web" containerID="cri-o://f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de" gracePeriod=600 Jun 09 03:04:32.805587 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:32.803346 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="thanos-sidecar" containerID="cri-o://2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567" gracePeriod=600 Jun 09 03:04:32.805587 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:32.803318 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy" containerID="cri-o://ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61" gracePeriod=600 Jun 09 03:04:33.546761 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546732 2564 generic.go:356] "Generic (PLEG): container finished" podID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerID="ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8" exitCode=0 Jun 09 03:04:33.546761 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546755 2564 generic.go:356] "Generic (PLEG): container finished" podID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerID="ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61" exitCode=0 Jun 09 03:04:33.546761 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546763 2564 generic.go:356] "Generic (PLEG): container finished" podID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerID="2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567" exitCode=0 Jun 09 03:04:33.546761 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546768 2564 generic.go:356] "Generic (PLEG): container finished" podID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerID="cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6" exitCode=0 Jun 09 03:04:33.547021 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546774 2564 generic.go:356] "Generic (PLEG): container finished" podID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerID="f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce" exitCode=0 Jun 09 03:04:33.547021 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546800 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerDied","Data":"ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8"} Jun 09 03:04:33.547021 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546857 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerDied","Data":"ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61"} Jun 09 03:04:33.547021 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546884 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerDied","Data":"2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567"} Jun 09 03:04:33.547021 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546906 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerDied","Data":"cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6"} Jun 09 03:04:33.547021 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:33.546930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerDied","Data":"f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce"} Jun 09 03:04:34.047374 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.047346 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.198207 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198184 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tdwb\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-kube-api-access-9tdwb\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198359 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198243 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198359 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198268 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-0\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198359 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198301 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-tls\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198359 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198331 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-grpc-tls\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198571 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198486 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-kubelet-serving-ca-bundle\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198647 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198626 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-serving-certs-ca-bundle\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198723 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198663 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198723 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198704 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198821 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198730 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-metrics-client-certs\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198821 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198757 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-1\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198821 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198789 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-db\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.198821 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198819 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config-out\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.199111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198844 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-thanos-prometheus-http-client-file\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.199111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198876 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-trusted-ca-bundle\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.199111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198901 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-2\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.199111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198933 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-web-config\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.199111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198925 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 03:04:34.199111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.198965 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-metrics-client-ca\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.199111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.199008 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-kube-rbac-proxy\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.199111 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.199038 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-tls-assets\") pod \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\" (UID: \"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a\") " Jun 09 03:04:34.199571 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.199314 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.199571 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.199445 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 03:04:34.199727 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.199708 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 03:04:34.200104 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.200014 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 03:04:34.200234 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.200157 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 03:04:34.200330 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.200307 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 03:04:34.200875 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.200829 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.200875 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.200849 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 03:04:34.201168 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.201142 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:04:34.201517 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.201492 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.201854 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.201803 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.202552 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.202506 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:04:34.202653 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.202573 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-kube-api-access-9tdwb" (OuterVolumeSpecName: "kube-api-access-9tdwb") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "kube-api-access-9tdwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:04:34.202940 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.202915 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.203054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.203029 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config-out" (OuterVolumeSpecName: "config-out") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:04:34.203905 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.203880 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.203905 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.203896 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config" (OuterVolumeSpecName: "config") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.204028 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.203914 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.204098 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.204076 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.214143 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.214100 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-web-config" (OuterVolumeSpecName: "web-config") pod "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" (UID: "f8659e0d-73c7-4e0b-95a5-0a8901bbf94a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:04:34.299946 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.299923 2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-grpc-tls\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.299946 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.299944 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.299955 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.299965 2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.299974 2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-metrics-client-certs\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.299982 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.299990 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-db\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.299997 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-config-out\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300005 2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300014 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300023 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300031 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-web-config\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300039 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-configmap-metrics-client-ca\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300047 2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-kube-rbac-proxy\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300056 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-tls-assets\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300446 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300064 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tdwb\" (UniqueName: \"kubernetes.io/projected/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-kube-api-access-9tdwb\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300446 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300073 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300446 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300081 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.300446 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.300089 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:04:34.553448 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.553348 2564 generic.go:356] "Generic (PLEG): container finished" podID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerID="f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de" exitCode=0 Jun 09 03:04:34.553448 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.553430 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.553448 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.553442 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerDied","Data":"f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de"} Jun 09 03:04:34.553722 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.553492 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8659e0d-73c7-4e0b-95a5-0a8901bbf94a","Type":"ContainerDied","Data":"d73143f02faf349eb9ef9b19a723b4408f674a7f77116f5848e368025b09f9cd"} Jun 09 03:04:34.553722 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.553525 2564 scope.go:117] "RemoveContainer" containerID="ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8" Jun 09 03:04:34.564574 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.564553 2564 scope.go:117] "RemoveContainer" containerID="ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61" Jun 09 03:04:34.571330 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.571312 2564 scope.go:117] "RemoveContainer" containerID="f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de" Jun 09 03:04:34.577864 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.577845 2564 scope.go:117] "RemoveContainer" containerID="2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567" Jun 09 03:04:34.579914 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.579896 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:04:34.584678 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.584660 2564 scope.go:117] "RemoveContainer" containerID="cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6" Jun 09 03:04:34.588507 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.588486 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:04:34.591621 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.591605 2564 scope.go:117] "RemoveContainer" containerID="f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce" Jun 09 03:04:34.598405 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.598392 2564 scope.go:117] "RemoveContainer" containerID="df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f" Jun 09 03:04:34.604384 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.604368 2564 scope.go:117] "RemoveContainer" containerID="ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8" Jun 09 03:04:34.604671 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:04:34.604642 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8\": container with ID starting with ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8 not found: ID does not exist" containerID="ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8" Jun 09 03:04:34.604724 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.604677 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8"} err="failed to get container status \"ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8\": rpc error: code = NotFound desc = could not find container \"ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8\": container with ID starting with ae162d6d6e039102dae8bb7b9757ae255ca565244c147ff5b19917475f2e30d8 not found: ID does not exist" Jun 09 03:04:34.604724 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.604708 2564 scope.go:117] "RemoveContainer" containerID="ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61" Jun 09 03:04:34.604914 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:04:34.604900 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61\": container with ID starting with ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61 not found: ID does not exist" containerID="ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61" Jun 09 03:04:34.604954 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.604917 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61"} err="failed to get container status \"ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61\": rpc error: code = NotFound desc = could not find container \"ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61\": container with ID starting with ce8ca1ac25b9a4b18c62ef584c8fdb10e52fbb608529e037230f85a59ba1ad61 not found: ID does not exist" Jun 09 03:04:34.604954 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.604929 2564 scope.go:117] "RemoveContainer" containerID="f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de" Jun 09 03:04:34.605163 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:04:34.605147 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de\": container with ID starting with f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de not found: ID does not exist" containerID="f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de" Jun 09 03:04:34.605230 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.605168 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de"} err="failed to get container status \"f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de\": rpc error: code = NotFound desc = could not find container \"f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de\": container with ID starting with f786f40ee3ae1ea0d442d3814540eb402ccbca95abe45ce419ec0da24e0709de not found: ID does not exist" Jun 09 03:04:34.605230 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.605183 2564 scope.go:117] "RemoveContainer" containerID="2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567" Jun 09 03:04:34.605442 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:04:34.605417 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567\": container with ID starting with 2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567 not found: ID does not exist" containerID="2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567" Jun 09 03:04:34.605496 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.605450 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567"} err="failed to get container status \"2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567\": rpc error: code = NotFound desc = could not find container \"2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567\": container with ID starting with 2ae694ec152c9b044c3967bfa1ee2711b9219e66ef8e9f48424dc8569f6b2567 not found: ID does not exist" Jun 09 03:04:34.605496 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.605465 2564 scope.go:117] "RemoveContainer" containerID="cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6" Jun 09 03:04:34.605852 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:04:34.605757 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6\": container with ID starting with cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6 not found: ID does not exist" containerID="cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6" Jun 09 03:04:34.605852 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.605785 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6"} err="failed to get container status \"cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6\": rpc error: code = NotFound desc = could not find container \"cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6\": container with ID starting with cce8a7e504642bf6d8d489ab2cc537f288f7259e7b53e7bc8f3ea03565101dc6 not found: ID does not exist" Jun 09 03:04:34.605852 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.605806 2564 scope.go:117] "RemoveContainer" containerID="f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce" Jun 09 03:04:34.606076 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:04:34.606055 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce\": container with ID starting with f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce not found: ID does not exist" containerID="f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce" Jun 09 03:04:34.606140 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.606081 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce"} err="failed to get container status \"f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce\": rpc error: code = NotFound desc = could not find container \"f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce\": container with ID starting with f257cd9bc393026463f539646de69317c611b4bbb3ef32d6cafed0b8978b1cce not found: ID does not exist" Jun 09 03:04:34.606140 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.606096 2564 scope.go:117] "RemoveContainer" containerID="df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f" Jun 09 03:04:34.606380 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:04:34.606350 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f\": container with ID starting with df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f not found: ID does not exist" containerID="df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f" Jun 09 03:04:34.606424 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.606379 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f"} err="failed to get container status \"df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f\": rpc error: code = NotFound desc = could not find container \"df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f\": container with ID starting with df79e9466e28c1e8d5f95a41a2f0420db8700ef80fa8b178c63e71e57dcaec9f not found: ID does not exist" Jun 09 03:04:34.613648 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613622 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:04:34.613912 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613897 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy-web" Jun 09 03:04:34.613983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613915 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy-web" Jun 09 03:04:34.613983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613929 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy" Jun 09 03:04:34.613983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613938 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy" Jun 09 03:04:34.613983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613949 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="init-config-reloader" Jun 09 03:04:34.613983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613959 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="init-config-reloader" Jun 09 03:04:34.613983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613973 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="prometheus" Jun 09 03:04:34.613983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613981 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="prometheus" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.613993 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy-thanos" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614001 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy-thanos" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614017 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="config-reloader" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614026 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="config-reloader" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614046 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="thanos-sidecar" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614057 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="thanos-sidecar" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614119 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="config-reloader" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614129 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614141 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy-web" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614152 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="prometheus" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614161 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="thanos-sidecar" Jun 09 03:04:34.614337 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.614169 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" containerName="kube-rbac-proxy-thanos" Jun 09 03:04:34.619326 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.619309 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.621522 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.621503 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 03:04:34.621655 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.621609 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 03:04:34.621779 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.621673 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 03:04:34.621779 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.621714 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 03:04:34.622005 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.621989 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 03:04:34.622097 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.621990 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 03:04:34.622298 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.622280 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 03:04:34.622634 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.622607 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 09 03:04:34.622744 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.622640 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 03:04:34.623329 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.623289 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 03:04:34.623423 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.623295 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 03:04:34.623423 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.623414 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ao4hlk6uvpt4g\"" Jun 09 03:04:34.623647 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.623566 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 03:04:34.623729 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.623667 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nnp87\"" Jun 09 03:04:34.623983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.623967 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 03:04:34.630995 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.630973 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 03:04:34.631943 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.631928 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 03:04:34.637109 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.637087 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:04:34.803020 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.802987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803020 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803021 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803188 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53d507e9-7e04-4da0-b18b-6b75a8743ee5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803188 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803066 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803188 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803116 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803188 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803166 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-config\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803221 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803239 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn9vm\" (UniqueName: \"kubernetes.io/projected/53d507e9-7e04-4da0-b18b-6b75a8743ee5-kube-api-access-wn9vm\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803256 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53d507e9-7e04-4da0-b18b-6b75a8743ee5-config-out\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803346 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803339 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803573 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803363 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-web-config\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803573 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803387 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803573 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803573 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803433 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803573 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803457 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803573 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803476 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.803573 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.803491 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904156 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904135 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904162 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53d507e9-7e04-4da0-b18b-6b75a8743ee5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904244 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-config\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904665 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904665 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn9vm\" (UniqueName: \"kubernetes.io/projected/53d507e9-7e04-4da0-b18b-6b75a8743ee5-kube-api-access-wn9vm\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904665 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53d507e9-7e04-4da0-b18b-6b75a8743ee5-config-out\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904665 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904532 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904665 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904556 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904665 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904585 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904665 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-web-config\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.904665 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904638 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.905047 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.905047 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904738 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.905047 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904761 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.905047 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.904786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.905263 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.905067 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.905369 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.905345 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.905548 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.905529 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.906112 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.906087 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.906927 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.906896 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53d507e9-7e04-4da0-b18b-6b75a8743ee5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.907091 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.907068 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.907240 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.907191 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.907350 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.907098 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.907939 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.907915 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-config\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.908020 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.907975 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.908020 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.908003 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.908837 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.908813 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53d507e9-7e04-4da0-b18b-6b75a8743ee5-config-out\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.909451 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.909222 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-web-config\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.909451 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.909400 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.909451 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.909433 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.909766 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.909747 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.909832 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.909782 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.910239 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.910224 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53d507e9-7e04-4da0-b18b-6b75a8743ee5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.911685 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.911662 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53d507e9-7e04-4da0-b18b-6b75a8743ee5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.912909 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.912893 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn9vm\" (UniqueName: \"kubernetes.io/projected/53d507e9-7e04-4da0-b18b-6b75a8743ee5-kube-api-access-wn9vm\") pod \"prometheus-k8s-0\" (UID: \"53d507e9-7e04-4da0-b18b-6b75a8743ee5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:34.933995 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:34.933971 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:04:35.084862 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:35.062049 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 03:04:35.084862 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:04:35.065744 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d507e9_7e04_4da0_b18b_6b75a8743ee5.slice/crio-01cc4de1082a1ae2b7d66b857697e2f78c28ea19043c187177f0fe2a814baa06 WatchSource:0}: Error finding container 01cc4de1082a1ae2b7d66b857697e2f78c28ea19043c187177f0fe2a814baa06: Status 404 returned error can't find the container with id 01cc4de1082a1ae2b7d66b857697e2f78c28ea19043c187177f0fe2a814baa06 Jun 09 03:04:35.558608 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:35.558573 2564 generic.go:356] "Generic (PLEG): container finished" podID="53d507e9-7e04-4da0-b18b-6b75a8743ee5" containerID="690ca845859d845d8d43160d45756b3cb9e4a58efb03463bb93066be535f199e" exitCode=0 Jun 09 03:04:35.558738 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:35.558667 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53d507e9-7e04-4da0-b18b-6b75a8743ee5","Type":"ContainerDied","Data":"690ca845859d845d8d43160d45756b3cb9e4a58efb03463bb93066be535f199e"} Jun 09 03:04:35.558738 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:35.558732 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53d507e9-7e04-4da0-b18b-6b75a8743ee5","Type":"ContainerStarted","Data":"01cc4de1082a1ae2b7d66b857697e2f78c28ea19043c187177f0fe2a814baa06"} Jun 09 03:04:36.549999 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:36.549967 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8659e0d-73c7-4e0b-95a5-0a8901bbf94a" path="/var/lib/kubelet/pods/f8659e0d-73c7-4e0b-95a5-0a8901bbf94a/volumes" Jun 09 03:04:36.566273 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:36.566239 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53d507e9-7e04-4da0-b18b-6b75a8743ee5","Type":"ContainerStarted","Data":"1c0d6f208f188f32994281788984f172684827db2dc843f7deddb8329bbc9a2e"} Jun 09 03:04:36.566406 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:36.566316 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53d507e9-7e04-4da0-b18b-6b75a8743ee5","Type":"ContainerStarted","Data":"e549b713718e47c18f02ea3e23dc5b17c338944a4146b129b32c2050031141da"} Jun 09 03:04:36.566406 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:36.566355 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53d507e9-7e04-4da0-b18b-6b75a8743ee5","Type":"ContainerStarted","Data":"5a478bfabefce67ec5d25930778b3d0a00b6acbd311b76e1db36b2e67464a327"} Jun 09 03:04:36.566406 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:36.566393 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53d507e9-7e04-4da0-b18b-6b75a8743ee5","Type":"ContainerStarted","Data":"215783c1a12ba8bd29611f1ce642beacce40953bfc8a90772aaaa2b0cc227178"} Jun 09 03:04:36.566540 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:36.566429 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53d507e9-7e04-4da0-b18b-6b75a8743ee5","Type":"ContainerStarted","Data":"3c18e8f54617f61bb6f71a52ac467de20f05a62c6ccee162821b5d7041b677f9"} Jun 09 03:04:36.566540 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:36.566463 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53d507e9-7e04-4da0-b18b-6b75a8743ee5","Type":"ContainerStarted","Data":"e8b29b1a9550caece563b52a83b23bcd8377725832639c64feb00b03570596df"} Jun 09 03:04:36.597371 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:36.597323 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.575060439 podStartE2EDuration="2.597309264s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.248825ms" isStatefulPod=false podCreationTimestamp="2026-06-09 03:04:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:04:36.595014769 +0000 UTC m=+310.504529116" watchObservedRunningTime="2026-06-09 03:04:36.597309264 +0000 UTC m=+310.506823610" Jun 09 03:04:39.935032 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:04:39.935000 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:05:34.934429 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:05:34.934391 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:05:34.949526 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:05:34.949500 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:05:35.789926 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:05:35.789901 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 03:08:02.338449 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.338422 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65"] Jun 09 03:08:02.342014 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.341998 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.343654 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.343627 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 09 03:08:02.343806 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.343685 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 09 03:08:02.343876 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.343859 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vl2w5\"" Jun 09 03:08:02.343920 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.343887 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 09 03:08:02.343963 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.343938 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 09 03:08:02.362611 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.362590 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65"] Jun 09 03:08:02.399674 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.399650 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10c1ac23-2fe7-44ab-9708-f46bed3d5662-tmp\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.399783 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.399688 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10c1ac23-2fe7-44ab-9708-f46bed3d5662-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.399783 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.399705 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10c1ac23-2fe7-44ab-9708-f46bed3d5662-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.399874 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.399790 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjzkj\" (UniqueName: \"kubernetes.io/projected/10c1ac23-2fe7-44ab-9708-f46bed3d5662-kube-api-access-jjzkj\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.399874 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.399818 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10c1ac23-2fe7-44ab-9708-f46bed3d5662-manifests\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.501152 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.501122 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10c1ac23-2fe7-44ab-9708-f46bed3d5662-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.501152 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.501151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10c1ac23-2fe7-44ab-9708-f46bed3d5662-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.501326 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.501236 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjzkj\" (UniqueName: \"kubernetes.io/projected/10c1ac23-2fe7-44ab-9708-f46bed3d5662-kube-api-access-jjzkj\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.501326 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.501292 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10c1ac23-2fe7-44ab-9708-f46bed3d5662-manifests\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.501408 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.501336 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10c1ac23-2fe7-44ab-9708-f46bed3d5662-tmp\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.501689 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.501672 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10c1ac23-2fe7-44ab-9708-f46bed3d5662-manifests\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.503371 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.503348 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10c1ac23-2fe7-44ab-9708-f46bed3d5662-tmp\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.503583 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.503565 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10c1ac23-2fe7-44ab-9708-f46bed3d5662-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.503674 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.503656 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10c1ac23-2fe7-44ab-9708-f46bed3d5662-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.507926 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.507907 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjzkj\" (UniqueName: \"kubernetes.io/projected/10c1ac23-2fe7-44ab-9708-f46bed3d5662-kube-api-access-jjzkj\") pod \"opendatahub-operator-controller-manager-86bfbb9599-x9x65\" (UID: \"10c1ac23-2fe7-44ab-9708-f46bed3d5662\") " pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.651686 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.651607 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:02.778374 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.778347 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65"] Jun 09 03:08:02.781038 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:08:02.781010 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c1ac23_2fe7_44ab_9708_f46bed3d5662.slice/crio-7d69ab64359c3c24fcdcee49ffe03a13fd4895d5542272289eba3d29bd396c70 WatchSource:0}: Error finding container 7d69ab64359c3c24fcdcee49ffe03a13fd4895d5542272289eba3d29bd396c70: Status 404 returned error can't find the container with id 7d69ab64359c3c24fcdcee49ffe03a13fd4895d5542272289eba3d29bd396c70 Jun 09 03:08:02.782805 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:02.782788 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 03:08:03.301228 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:03.301181 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" event={"ID":"10c1ac23-2fe7-44ab-9708-f46bed3d5662","Type":"ContainerStarted","Data":"7d69ab64359c3c24fcdcee49ffe03a13fd4895d5542272289eba3d29bd396c70"} Jun 09 03:08:06.317175 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:06.317139 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" event={"ID":"10c1ac23-2fe7-44ab-9708-f46bed3d5662","Type":"ContainerStarted","Data":"83e40cdac93e67f16f13f7c9b39d887b7ebb97137acf11fcb566ff214e3eb3ac"} Jun 09 03:08:07.322128 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:07.322092 2564 generic.go:356] "Generic (PLEG): container finished" podID="10c1ac23-2fe7-44ab-9708-f46bed3d5662" containerID="83e40cdac93e67f16f13f7c9b39d887b7ebb97137acf11fcb566ff214e3eb3ac" exitCode=0 Jun 09 03:08:07.322497 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:07.322146 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" event={"ID":"10c1ac23-2fe7-44ab-9708-f46bed3d5662","Type":"ContainerDied","Data":"83e40cdac93e67f16f13f7c9b39d887b7ebb97137acf11fcb566ff214e3eb3ac"} Jun 09 03:08:08.327494 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:08.327456 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" event={"ID":"10c1ac23-2fe7-44ab-9708-f46bed3d5662","Type":"ContainerStarted","Data":"fd2cfa35f59280e693067685320810866e708e5aadcb07a622979256ca3c1034"} Jun 09 03:08:08.327955 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:08.327565 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:08.348416 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:08.348357 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" podStartSLOduration=2.258343165 podStartE2EDuration="6.34833837s" totalImagesPullingTime="3.145815719s" totalInitContainerRuntime="944.179486ms" isStatefulPod=false podCreationTimestamp="2026-06-09 03:08:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:08:08.346623843 +0000 UTC m=+522.256138189" watchObservedRunningTime="2026-06-09 03:08:08.34833837 +0000 UTC m=+522.257852717" Jun 09 03:08:19.333499 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:19.333427 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86bfbb9599-x9x65" Jun 09 03:08:29.124015 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.123982 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28"] Jun 09 03:08:29.131313 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.131294 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.134516 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.134284 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 09 03:08:29.134516 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.134285 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 09 03:08:29.134516 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.134351 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 09 03:08:29.134516 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.134366 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 09 03:08:29.134516 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.134410 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 09 03:08:29.134516 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.134410 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bpq4l\"" Jun 09 03:08:29.135612 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.135591 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28"] Jun 09 03:08:29.214705 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.214673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2d2803b7-6583-4a0c-bb4a-666d4377b47b-manager-config\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.214871 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.214713 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d2803b7-6583-4a0c-bb4a-666d4377b47b-cert\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.214871 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.214794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5b99\" (UniqueName: \"kubernetes.io/projected/2d2803b7-6583-4a0c-bb4a-666d4377b47b-kube-api-access-p5b99\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.214871 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.214820 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d2803b7-6583-4a0c-bb4a-666d4377b47b-metrics-cert\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.315732 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.315696 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2d2803b7-6583-4a0c-bb4a-666d4377b47b-manager-config\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.315732 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.315738 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d2803b7-6583-4a0c-bb4a-666d4377b47b-cert\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.315959 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.315791 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5b99\" (UniqueName: \"kubernetes.io/projected/2d2803b7-6583-4a0c-bb4a-666d4377b47b-kube-api-access-p5b99\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.315959 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.315814 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d2803b7-6583-4a0c-bb4a-666d4377b47b-metrics-cert\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.316435 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.316413 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2d2803b7-6583-4a0c-bb4a-666d4377b47b-manager-config\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.318299 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.318279 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d2803b7-6583-4a0c-bb4a-666d4377b47b-cert\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.318443 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.318422 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d2803b7-6583-4a0c-bb4a-666d4377b47b-metrics-cert\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.325754 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.325732 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5b99\" (UniqueName: \"kubernetes.io/projected/2d2803b7-6583-4a0c-bb4a-666d4377b47b-kube-api-access-p5b99\") pod \"lws-controller-manager-59599dfcbf-n6r28\" (UID: \"2d2803b7-6583-4a0c-bb4a-666d4377b47b\") " pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.442323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.442231 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:29.582602 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:29.582579 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28"] Jun 09 03:08:29.586465 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:08:29.586439 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d2803b7_6583_4a0c_bb4a_666d4377b47b.slice/crio-ad0f096c46834e050c233ad95cfb07782a17fbdf976ea67b4a201cce9b787c0a WatchSource:0}: Error finding container ad0f096c46834e050c233ad95cfb07782a17fbdf976ea67b4a201cce9b787c0a: Status 404 returned error can't find the container with id ad0f096c46834e050c233ad95cfb07782a17fbdf976ea67b4a201cce9b787c0a Jun 09 03:08:30.421181 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:30.421132 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" event={"ID":"2d2803b7-6583-4a0c-bb4a-666d4377b47b","Type":"ContainerStarted","Data":"ad0f096c46834e050c233ad95cfb07782a17fbdf976ea67b4a201cce9b787c0a"} Jun 09 03:08:32.431178 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:32.431085 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" event={"ID":"2d2803b7-6583-4a0c-bb4a-666d4377b47b","Type":"ContainerStarted","Data":"b3986da59b6942e921a84e4609b614b8a848298280232d93c18ec88bd4f2195b"} Jun 09 03:08:32.431661 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:32.431233 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:32.448289 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:32.448230 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" podStartSLOduration=0.909719593 podStartE2EDuration="3.448191278s" totalImagesPullingTime="2.538471685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:08:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:08:32.446924274 +0000 UTC m=+546.356438647" watchObservedRunningTime="2026-06-09 03:08:32.448191278 +0000 UTC m=+546.357705705" Jun 09 03:08:43.438181 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:43.438145 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59599dfcbf-n6r28" Jun 09 03:08:54.841534 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.841494 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn"] Jun 09 03:08:54.844036 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.844005 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.846037 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.846017 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 09 03:08:54.846150 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.846120 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ssmzj\"" Jun 09 03:08:54.854597 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.854576 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn"] Jun 09 03:08:54.933029 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.932995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.933183 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.933036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.933183 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.933060 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.933183 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.933136 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.933324 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.933262 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.933324 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.933301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.933387 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.933342 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.933422 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.933373 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:54.933455 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:54.933417 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ttvh\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-kube-api-access-7ttvh\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.033769 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033740 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.033948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033775 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.033948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033800 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.033948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033818 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.033948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033836 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ttvh\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-kube-api-access-7ttvh\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.033948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033865 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.033948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033898 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.033948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.034396 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.033954 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.034454 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.034406 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.034454 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.034434 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.034599 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.034579 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.034663 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.034601 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.034663 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.034619 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.036330 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.036308 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.036936 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.036913 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.041908 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.041876 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ttvh\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-kube-api-access-7ttvh\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.042093 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.042079 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.149995 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.149921 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6"] Jun 09 03:08:55.153326 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.153309 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.154290 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.154270 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:55.169818 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.169790 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6"] Jun 09 03:08:55.235646 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235610 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89f322a8-4802-4cb4-b690-4a89dc3deed0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.235814 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zfzh\" (UniqueName: \"kubernetes.io/projected/89f322a8-4802-4cb4-b690-4a89dc3deed0-kube-api-access-7zfzh\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.235814 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.235814 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.235814 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.235814 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235813 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.236088 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235832 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.236088 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235853 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.236088 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.235923 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.298518 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.298494 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn"] Jun 09 03:08:55.301239 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:08:55.301193 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d8b22d_9dd1_4c10_bffd_8c902c989e92.slice/crio-058b809a2bfbcfd4310bd586ae8acd4015243f39fc1fa4a7e6b84b3668518db6 WatchSource:0}: Error finding container 058b809a2bfbcfd4310bd586ae8acd4015243f39fc1fa4a7e6b84b3668518db6: Status 404 returned error can't find the container with id 058b809a2bfbcfd4310bd586ae8acd4015243f39fc1fa4a7e6b84b3668518db6 Jun 09 03:08:55.336647 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.336613 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.336768 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.336652 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.336768 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.336686 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.336768 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.336737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.336934 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.336772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89f322a8-4802-4cb4-b690-4a89dc3deed0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.336934 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.336816 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zfzh\" (UniqueName: \"kubernetes.io/projected/89f322a8-4802-4cb4-b690-4a89dc3deed0-kube-api-access-7zfzh\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.336934 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.336843 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.337045 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.336971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.337045 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.337006 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.337045 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.337037 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.337375 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.337352 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.337525 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.337502 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.337636 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.337370 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.337691 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.337543 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89f322a8-4802-4cb4-b690-4a89dc3deed0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.339112 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.339094 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.339261 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.339245 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.345852 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.345831 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89f322a8-4802-4cb4-b690-4a89dc3deed0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.346034 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.346018 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zfzh\" (UniqueName: \"kubernetes.io/projected/89f322a8-4802-4cb4-b690-4a89dc3deed0-kube-api-access-7zfzh\") pod \"data-science-gateway-data-science-gateway-class-74759f979544fr6\" (UID: \"89f322a8-4802-4cb4-b690-4a89dc3deed0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.479104 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.479072 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:55.521512 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.521481 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" event={"ID":"59d8b22d-9dd1-4c10-bffd-8c902c989e92","Type":"ContainerStarted","Data":"058b809a2bfbcfd4310bd586ae8acd4015243f39fc1fa4a7e6b84b3668518db6"} Jun 09 03:08:55.649127 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:55.648992 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6"] Jun 09 03:08:55.651646 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:08:55.651621 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f322a8_4802_4cb4_b690_4a89dc3deed0.slice/crio-f2d33f89bf56bb24e7f2d6262097ded37846b5ec91a3d48addfaf07cb70d351f WatchSource:0}: Error finding container f2d33f89bf56bb24e7f2d6262097ded37846b5ec91a3d48addfaf07cb70d351f: Status 404 returned error can't find the container with id f2d33f89bf56bb24e7f2d6262097ded37846b5ec91a3d48addfaf07cb70d351f Jun 09 03:08:56.529699 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:56.529658 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" event={"ID":"89f322a8-4802-4cb4-b690-4a89dc3deed0","Type":"ContainerStarted","Data":"f2d33f89bf56bb24e7f2d6262097ded37846b5ec91a3d48addfaf07cb70d351f"} Jun 09 03:08:58.048954 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.048911 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:08:58.049254 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.048985 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:08:58.049254 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.049013 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:08:58.054425 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.054393 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:08:58.054503 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.054465 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:08:58.054540 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.054512 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:08:58.539692 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.539656 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" event={"ID":"59d8b22d-9dd1-4c10-bffd-8c902c989e92","Type":"ContainerStarted","Data":"41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10"} Jun 09 03:08:58.541162 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.541139 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" event={"ID":"89f322a8-4802-4cb4-b690-4a89dc3deed0","Type":"ContainerStarted","Data":"1c1b76b349834b4d61b56a712af1f3f3e2a77216338312097abfee6da845078a"} Jun 09 03:08:58.557692 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.557642 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" podStartSLOduration=1.812199042 podStartE2EDuration="4.557623767s" totalImagesPullingTime="2.745424725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:08:58.556179778 +0000 UTC m=+572.465694125" watchObservedRunningTime="2026-06-09 03:08:58.557623767 +0000 UTC m=+572.467138115" Jun 09 03:08:58.572587 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.572546 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" podStartSLOduration=1.172030166 podStartE2EDuration="3.572531753s" totalImagesPullingTime="2.400501587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:08:58.571256166 +0000 UTC m=+572.480770513" watchObservedRunningTime="2026-06-09 03:08:58.572531753 +0000 UTC m=+572.482046099" Jun 09 03:08:58.919619 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.919541 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-6d869d9c69-mpkfv"] Jun 09 03:08:58.922186 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.922165 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" Jun 09 03:08:58.924222 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.924189 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 09 03:08:58.924328 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.924308 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ffkzp\"" Jun 09 03:08:58.933574 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.933547 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6d869d9c69-mpkfv"] Jun 09 03:08:58.972143 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.972104 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssrsk\" (UniqueName: \"kubernetes.io/projected/42295f07-c81a-444a-858e-f6a175e65a4a-kube-api-access-ssrsk\") pod \"dashboard-redirect-6d869d9c69-mpkfv\" (UID: \"42295f07-c81a-444a-858e-f6a175e65a4a\") " pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" Jun 09 03:08:58.972333 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:58.972171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/42295f07-c81a-444a-858e-f6a175e65a4a-redirect-config\") pod \"dashboard-redirect-6d869d9c69-mpkfv\" (UID: \"42295f07-c81a-444a-858e-f6a175e65a4a\") " pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" Jun 09 03:08:59.073124 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.073087 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssrsk\" (UniqueName: \"kubernetes.io/projected/42295f07-c81a-444a-858e-f6a175e65a4a-kube-api-access-ssrsk\") pod \"dashboard-redirect-6d869d9c69-mpkfv\" (UID: \"42295f07-c81a-444a-858e-f6a175e65a4a\") " pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" Jun 09 03:08:59.073614 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.073158 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/42295f07-c81a-444a-858e-f6a175e65a4a-redirect-config\") pod \"dashboard-redirect-6d869d9c69-mpkfv\" (UID: \"42295f07-c81a-444a-858e-f6a175e65a4a\") " pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" Jun 09 03:08:59.073816 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.073797 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/42295f07-c81a-444a-858e-f6a175e65a4a-redirect-config\") pod \"dashboard-redirect-6d869d9c69-mpkfv\" (UID: \"42295f07-c81a-444a-858e-f6a175e65a4a\") " pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" Jun 09 03:08:59.081322 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.081297 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssrsk\" (UniqueName: \"kubernetes.io/projected/42295f07-c81a-444a-858e-f6a175e65a4a-kube-api-access-ssrsk\") pod \"dashboard-redirect-6d869d9c69-mpkfv\" (UID: \"42295f07-c81a-444a-858e-f6a175e65a4a\") " pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" Jun 09 03:08:59.154623 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.154594 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:08:59.156157 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.156133 2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Jun 09 03:08:59.156292 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.156181 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" podUID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Jun 09 03:08:59.233495 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.233466 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" Jun 09 03:08:59.356373 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.356349 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6d869d9c69-mpkfv"] Jun 09 03:08:59.358715 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:08:59.358689 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42295f07_c81a_444a_858e_f6a175e65a4a.slice/crio-8bd678dd27cb8eefcdbd3c1f0743e3b72011efdddde9d64868bccd094a1f2d2c WatchSource:0}: Error finding container 8bd678dd27cb8eefcdbd3c1f0743e3b72011efdddde9d64868bccd094a1f2d2c: Status 404 returned error can't find the container with id 8bd678dd27cb8eefcdbd3c1f0743e3b72011efdddde9d64868bccd094a1f2d2c Jun 09 03:08:59.479427 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.479394 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:59.484500 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.484434 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:59.546871 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.546837 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" event={"ID":"42295f07-c81a-444a-858e-f6a175e65a4a","Type":"ContainerStarted","Data":"8bd678dd27cb8eefcdbd3c1f0743e3b72011efdddde9d64868bccd094a1f2d2c"} Jun 09 03:08:59.547190 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.547173 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:59.548108 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.548088 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979544fr6" Jun 09 03:08:59.590845 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:08:59.590807 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn"] Jun 09 03:09:00.154983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:00.154948 2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Jun 09 03:09:00.155464 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:00.155013 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" podUID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Jun 09 03:09:01.155094 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:01.155057 2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Jun 09 03:09:01.155504 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:01.155125 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" podUID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Jun 09 03:09:01.557043 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:01.557017 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" podUID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" containerName="istio-proxy" containerID="cri-o://41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10" gracePeriod=30 Jun 09 03:09:02.562685 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:02.562642 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" event={"ID":"42295f07-c81a-444a-858e-f6a175e65a4a","Type":"ContainerStarted","Data":"a590aac86c594c6218bfb2716d13bdbeb0c2d931d9fbf8c42f046026a35d7ab2"} Jun 09 03:09:02.578183 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:02.578133 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-6d869d9c69-mpkfv" podStartSLOduration=2.41567414 podStartE2EDuration="4.578115645s" totalImagesPullingTime="2.162441505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:09:02.576527284 +0000 UTC m=+576.486041631" watchObservedRunningTime="2026-06-09 03:09:02.578115645 +0000 UTC m=+576.487629992" Jun 09 03:09:06.810951 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.810927 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:09:06.840801 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840709 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-certs\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.840801 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840759 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-podinfo\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.840801 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840787 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-token\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.841085 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840820 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-data\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.841085 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840839 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-envoy\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.841085 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840894 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-socket\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.841085 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840920 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istiod-ca-cert\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.841085 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840957 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-credential-socket\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.841085 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.840982 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ttvh\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-kube-api-access-7ttvh\") pod \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\" (UID: \"59d8b22d-9dd1-4c10-bffd-8c902c989e92\") " Jun 09 03:09:06.841442 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.841135 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-data" (OuterVolumeSpecName: "istio-data") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:09:06.841442 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.841292 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-data\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:06.841442 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.841348 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:09:06.841600 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.841503 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:09:06.841857 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.841802 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 03:09:06.842247 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.842228 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:09:06.843671 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.843643 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-token" (OuterVolumeSpecName: "istio-token") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:09:06.844255 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.844233 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:09:06.844352 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.844285 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-kube-api-access-7ttvh" (OuterVolumeSpecName: "kube-api-access-7ttvh") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "kube-api-access-7ttvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:09:06.844352 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.844305 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59d8b22d-9dd1-4c10-bffd-8c902c989e92" (UID: "59d8b22d-9dd1-4c10-bffd-8c902c989e92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 09 03:09:06.942068 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.942035 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-certs\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:06.942068 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.942061 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-podinfo\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:06.942068 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.942070 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-token\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:06.942068 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.942078 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istio-envoy\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:06.942336 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.942086 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-workload-socket\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:06.942336 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.942094 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d8b22d-9dd1-4c10-bffd-8c902c989e92-istiod-ca-cert\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:06.942336 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.942102 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d8b22d-9dd1-4c10-bffd-8c902c989e92-credential-socket\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:06.942336 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:06.942110 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ttvh\" (UniqueName: \"kubernetes.io/projected/59d8b22d-9dd1-4c10-bffd-8c902c989e92-kube-api-access-7ttvh\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:07.583644 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.583610 2564 generic.go:356] "Generic (PLEG): container finished" podID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" containerID="41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10" exitCode=0 Jun 09 03:09:07.583644 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.583649 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" event={"ID":"59d8b22d-9dd1-4c10-bffd-8c902c989e92","Type":"ContainerDied","Data":"41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10"} Jun 09 03:09:07.583897 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.583657 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" Jun 09 03:09:07.583897 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.583690 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn" event={"ID":"59d8b22d-9dd1-4c10-bffd-8c902c989e92","Type":"ContainerDied","Data":"058b809a2bfbcfd4310bd586ae8acd4015243f39fc1fa4a7e6b84b3668518db6"} Jun 09 03:09:07.583897 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.583709 2564 scope.go:117] "RemoveContainer" containerID="41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10" Jun 09 03:09:07.592681 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.592664 2564 scope.go:117] "RemoveContainer" containerID="41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10" Jun 09 03:09:07.592923 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:09:07.592904 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10\": container with ID starting with 41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10 not found: ID does not exist" containerID="41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10" Jun 09 03:09:07.592983 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.592930 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10"} err="failed to get container status \"41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10\": rpc error: code = NotFound desc = could not find container \"41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10\": container with ID starting with 41d0db594128232f437fb0f750536f74de6e92c7863314d83aa119efd0ef3e10 not found: ID does not exist" Jun 09 03:09:07.603304 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.603279 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn"] Jun 09 03:09:07.615538 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:07.613256 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6f9hrmn"] Jun 09 03:09:08.549865 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:08.549834 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" path="/var/lib/kubelet/pods/59d8b22d-9dd1-4c10-bffd-8c902c989e92/volumes" Jun 09 03:09:22.634262 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.634231 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2679"] Jun 09 03:09:22.634690 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.634531 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" containerName="istio-proxy" Jun 09 03:09:22.634690 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.634541 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" containerName="istio-proxy" Jun 09 03:09:22.634690 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.634616 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="59d8b22d-9dd1-4c10-bffd-8c902c989e92" containerName="istio-proxy" Jun 09 03:09:22.636625 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.636605 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2679" Jun 09 03:09:22.638692 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.638669 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 09 03:09:22.639067 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.639050 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ffwt6\"" Jun 09 03:09:22.639154 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.639055 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 09 03:09:22.644759 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.644730 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2679"] Jun 09 03:09:22.784602 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.784574 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw46t\" (UniqueName: \"kubernetes.io/projected/db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb-kube-api-access-tw46t\") pod \"kuadrant-operator-catalog-h2679\" (UID: \"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb\") " pod="kuadrant-system/kuadrant-operator-catalog-h2679" Jun 09 03:09:22.885828 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.885744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw46t\" (UniqueName: \"kubernetes.io/projected/db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb-kube-api-access-tw46t\") pod \"kuadrant-operator-catalog-h2679\" (UID: \"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb\") " pod="kuadrant-system/kuadrant-operator-catalog-h2679" Jun 09 03:09:22.898296 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.898267 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw46t\" (UniqueName: \"kubernetes.io/projected/db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb-kube-api-access-tw46t\") pod \"kuadrant-operator-catalog-h2679\" (UID: \"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb\") " pod="kuadrant-system/kuadrant-operator-catalog-h2679" Jun 09 03:09:22.949050 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:22.949019 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2679" Jun 09 03:09:23.008636 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.008589 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2679"] Jun 09 03:09:23.073650 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.073625 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2679"] Jun 09 03:09:23.076028 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:09:23.076001 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb18c7f0_2f3c_4a90_8a68_0f2ec419d4eb.slice/crio-d8c2a0306d55bda521b86f1ba7747720435d8d651aa5e62ed3c344d969a46d8f WatchSource:0}: Error finding container d8c2a0306d55bda521b86f1ba7747720435d8d651aa5e62ed3c344d969a46d8f: Status 404 returned error can't find the container with id d8c2a0306d55bda521b86f1ba7747720435d8d651aa5e62ed3c344d969a46d8f Jun 09 03:09:23.216287 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.216255 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9s4bt"] Jun 09 03:09:23.219345 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.219329 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:23.227653 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.227631 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9s4bt"] Jun 09 03:09:23.290406 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.290377 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shgqm\" (UniqueName: \"kubernetes.io/projected/b1970ad6-7faf-4ec2-8351-6660e1a96495-kube-api-access-shgqm\") pod \"kuadrant-operator-catalog-9s4bt\" (UID: \"b1970ad6-7faf-4ec2-8351-6660e1a96495\") " pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:23.391191 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.391156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shgqm\" (UniqueName: \"kubernetes.io/projected/b1970ad6-7faf-4ec2-8351-6660e1a96495-kube-api-access-shgqm\") pod \"kuadrant-operator-catalog-9s4bt\" (UID: \"b1970ad6-7faf-4ec2-8351-6660e1a96495\") " pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:23.398915 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.398888 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shgqm\" (UniqueName: \"kubernetes.io/projected/b1970ad6-7faf-4ec2-8351-6660e1a96495-kube-api-access-shgqm\") pod \"kuadrant-operator-catalog-9s4bt\" (UID: \"b1970ad6-7faf-4ec2-8351-6660e1a96495\") " pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:23.529509 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.529416 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:23.655515 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.655459 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2679" event={"ID":"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb","Type":"ContainerStarted","Data":"d8c2a0306d55bda521b86f1ba7747720435d8d651aa5e62ed3c344d969a46d8f"} Jun 09 03:09:23.658962 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:23.658938 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9s4bt"] Jun 09 03:09:23.661256 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:09:23.661231 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1970ad6_7faf_4ec2_8351_6660e1a96495.slice/crio-1f239e73fc16bc28985c0571a257d6d162f863be530cfadfe18e61e666fd8d22 WatchSource:0}: Error finding container 1f239e73fc16bc28985c0571a257d6d162f863be530cfadfe18e61e666fd8d22: Status 404 returned error can't find the container with id 1f239e73fc16bc28985c0571a257d6d162f863be530cfadfe18e61e666fd8d22 Jun 09 03:09:24.661652 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:24.661606 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" event={"ID":"b1970ad6-7faf-4ec2-8351-6660e1a96495","Type":"ContainerStarted","Data":"1f239e73fc16bc28985c0571a257d6d162f863be530cfadfe18e61e666fd8d22"} Jun 09 03:09:25.667076 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:25.666981 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" event={"ID":"b1970ad6-7faf-4ec2-8351-6660e1a96495","Type":"ContainerStarted","Data":"4e80dee65aea82136e99d63be7069515d5b2a532e8974e7de7b6bd5d732a4291"} Jun 09 03:09:25.668488 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:25.668463 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2679" event={"ID":"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb","Type":"ContainerStarted","Data":"71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0"} Jun 09 03:09:25.668624 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:25.668491 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-h2679" podUID="db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb" containerName="registry-server" containerID="cri-o://71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0" gracePeriod=2 Jun 09 03:09:25.682128 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:25.682081 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" podStartSLOduration=1.061902609 podStartE2EDuration="2.682066775s" totalImagesPullingTime="1.620164166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:09:25.680612778 +0000 UTC m=+599.590127125" watchObservedRunningTime="2026-06-09 03:09:25.682066775 +0000 UTC m=+599.591581118" Jun 09 03:09:25.697550 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:25.697508 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h2679" podStartSLOduration=1.4958528229999999 podStartE2EDuration="3.697491901s" totalImagesPullingTime="2.201639078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:09:25.696336757 +0000 UTC m=+599.605851104" watchObservedRunningTime="2026-06-09 03:09:25.697491901 +0000 UTC m=+599.607006269" Jun 09 03:09:25.916568 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:25.916542 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2679" Jun 09 03:09:26.017741 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.017704 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw46t\" (UniqueName: \"kubernetes.io/projected/db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb-kube-api-access-tw46t\") pod \"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb\" (UID: \"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb\") " Jun 09 03:09:26.019934 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.019897 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb-kube-api-access-tw46t" (OuterVolumeSpecName: "kube-api-access-tw46t") pod "db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb" (UID: "db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb"). InnerVolumeSpecName "kube-api-access-tw46t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:09:26.119332 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.119299 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw46t\" (UniqueName: \"kubernetes.io/projected/db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb-kube-api-access-tw46t\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:09:26.673570 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.673526 2564 generic.go:356] "Generic (PLEG): container finished" podID="db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb" containerID="71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0" exitCode=0 Jun 09 03:09:26.674033 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.673585 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2679" Jun 09 03:09:26.674033 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.673613 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2679" event={"ID":"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb","Type":"ContainerDied","Data":"71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0"} Jun 09 03:09:26.674033 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.673696 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2679" event={"ID":"db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb","Type":"ContainerDied","Data":"d8c2a0306d55bda521b86f1ba7747720435d8d651aa5e62ed3c344d969a46d8f"} Jun 09 03:09:26.674033 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.673726 2564 scope.go:117] "RemoveContainer" containerID="71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0" Jun 09 03:09:26.682503 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.682487 2564 scope.go:117] "RemoveContainer" containerID="71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0" Jun 09 03:09:26.682765 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:09:26.682740 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0\": container with ID starting with 71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0 not found: ID does not exist" containerID="71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0" Jun 09 03:09:26.682846 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.682767 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0"} err="failed to get container status \"71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0\": rpc error: code = NotFound desc = could not find container \"71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0\": container with ID starting with 71eea7833f3edca861b16f0a4569d7d19405a955a609238f0b881c88aaed7dc0 not found: ID does not exist" Jun 09 03:09:26.686695 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.686671 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2679"] Jun 09 03:09:26.690052 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:26.690031 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2679"] Jun 09 03:09:28.550462 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:28.550433 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb" path="/var/lib/kubelet/pods/db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb/volumes" Jun 09 03:09:33.530434 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:33.530399 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:33.531757 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:33.530456 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:33.552265 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:33.552243 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:33.722806 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:33.722780 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-9s4bt" Jun 09 03:09:54.371908 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.371821 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52"] Jun 09 03:09:54.372363 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.372177 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb" containerName="registry-server" Jun 09 03:09:54.372363 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.372191 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb" containerName="registry-server" Jun 09 03:09:54.372363 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.372278 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="db18c7f0-2f3c-4a90-8a68-0f2ec419d4eb" containerName="registry-server" Jun 09 03:09:54.375186 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.375168 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" Jun 09 03:09:54.377129 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.377106 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 09 03:09:54.377306 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.377231 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-t5jdz\"" Jun 09 03:09:54.386495 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.386460 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52"] Jun 09 03:09:54.461709 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.461665 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc47d\" (UniqueName: \"kubernetes.io/projected/270883d6-8776-4d80-b736-b71c9d96a2b7-kube-api-access-zc47d\") pod \"dns-operator-controller-manager-864dd78c4c-j5d52\" (UID: \"270883d6-8776-4d80-b736-b71c9d96a2b7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" Jun 09 03:09:54.562637 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.562596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc47d\" (UniqueName: \"kubernetes.io/projected/270883d6-8776-4d80-b736-b71c9d96a2b7-kube-api-access-zc47d\") pod \"dns-operator-controller-manager-864dd78c4c-j5d52\" (UID: \"270883d6-8776-4d80-b736-b71c9d96a2b7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" Jun 09 03:09:54.574683 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.574645 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc47d\" (UniqueName: \"kubernetes.io/projected/270883d6-8776-4d80-b736-b71c9d96a2b7-kube-api-access-zc47d\") pod \"dns-operator-controller-manager-864dd78c4c-j5d52\" (UID: \"270883d6-8776-4d80-b736-b71c9d96a2b7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" Jun 09 03:09:54.687558 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.687468 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" Jun 09 03:09:54.832441 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:54.832399 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52"] Jun 09 03:09:54.835195 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:09:54.835164 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod270883d6_8776_4d80_b736_b71c9d96a2b7.slice/crio-030b2a0952a02c5d4c865c1f462abef84918cf17b144dd3e41508aa351c2b417 WatchSource:0}: Error finding container 030b2a0952a02c5d4c865c1f462abef84918cf17b144dd3e41508aa351c2b417: Status 404 returned error can't find the container with id 030b2a0952a02c5d4c865c1f462abef84918cf17b144dd3e41508aa351c2b417 Jun 09 03:09:55.797178 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:55.797135 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" event={"ID":"270883d6-8776-4d80-b736-b71c9d96a2b7","Type":"ContainerStarted","Data":"030b2a0952a02c5d4c865c1f462abef84918cf17b144dd3e41508aa351c2b417"} Jun 09 03:09:57.807982 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:57.807940 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" event={"ID":"270883d6-8776-4d80-b736-b71c9d96a2b7","Type":"ContainerStarted","Data":"b4792a064065147b197c69441cd85e1df5b27a35f5e153b28e369aaddfeab641"} Jun 09 03:09:57.808451 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:57.808040 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" Jun 09 03:09:57.824278 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:57.824185 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" podStartSLOduration=1.252588146 podStartE2EDuration="3.824155959s" totalImagesPullingTime="2.571567813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:09:57.823643601 +0000 UTC m=+631.733157949" watchObservedRunningTime="2026-06-09 03:09:57.824155959 +0000 UTC m=+631.733670307" Jun 09 03:09:57.864134 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:57.864107 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg"] Jun 09 03:09:57.867592 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:57.867558 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:09:57.870106 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:57.870064 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hbj9r\"" Jun 09 03:09:57.891425 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:57.891341 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg"] Jun 09 03:09:57.994695 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:57.994655 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vhgb\" (UniqueName: \"kubernetes.io/projected/596169dd-de1f-445a-8e81-4c3d0670d6cf-kube-api-access-9vhgb\") pod \"limitador-operator-controller-manager-859d7b78bd-cngbg\" (UID: \"596169dd-de1f-445a-8e81-4c3d0670d6cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:09:58.095838 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:58.095759 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vhgb\" (UniqueName: \"kubernetes.io/projected/596169dd-de1f-445a-8e81-4c3d0670d6cf-kube-api-access-9vhgb\") pod \"limitador-operator-controller-manager-859d7b78bd-cngbg\" (UID: \"596169dd-de1f-445a-8e81-4c3d0670d6cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:09:58.103595 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:58.103567 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vhgb\" (UniqueName: \"kubernetes.io/projected/596169dd-de1f-445a-8e81-4c3d0670d6cf-kube-api-access-9vhgb\") pod \"limitador-operator-controller-manager-859d7b78bd-cngbg\" (UID: \"596169dd-de1f-445a-8e81-4c3d0670d6cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:09:58.186794 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:58.186737 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:09:58.336326 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:58.336292 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg"] Jun 09 03:09:58.339281 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:09:58.339252 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596169dd_de1f_445a_8e81_4c3d0670d6cf.slice/crio-93b64a90fb638b4fc75362a3e7a6ad431203159e01fcb7ab7e3a090d3101e0a9 WatchSource:0}: Error finding container 93b64a90fb638b4fc75362a3e7a6ad431203159e01fcb7ab7e3a090d3101e0a9: Status 404 returned error can't find the container with id 93b64a90fb638b4fc75362a3e7a6ad431203159e01fcb7ab7e3a090d3101e0a9 Jun 09 03:09:58.814513 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:09:58.814468 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" event={"ID":"596169dd-de1f-445a-8e81-4c3d0670d6cf","Type":"ContainerStarted","Data":"93b64a90fb638b4fc75362a3e7a6ad431203159e01fcb7ab7e3a090d3101e0a9"} Jun 09 03:10:00.825384 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:00.825282 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" event={"ID":"596169dd-de1f-445a-8e81-4c3d0670d6cf","Type":"ContainerStarted","Data":"86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1"} Jun 09 03:10:00.825876 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:00.825417 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:00.840106 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:00.840037 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" podStartSLOduration=1.6115927509999999 podStartE2EDuration="3.840018634s" totalImagesPullingTime="2.228425883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:10:00.838331624 +0000 UTC m=+634.747845995" watchObservedRunningTime="2026-06-09 03:10:00.840018634 +0000 UTC m=+634.749532977" Jun 09 03:10:06.399302 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.399260 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv"] Jun 09 03:10:06.402873 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.402854 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:06.404976 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.404953 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qn28q\"" Jun 09 03:10:06.414387 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.414360 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv"] Jun 09 03:10:06.478008 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.477968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d37144-0815-441e-a09f-83037562c511-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" (UID: \"e9d37144-0815-441e-a09f-83037562c511\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:06.478229 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.478038 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55rd7\" (UniqueName: \"kubernetes.io/projected/e9d37144-0815-441e-a09f-83037562c511-kube-api-access-55rd7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" (UID: \"e9d37144-0815-441e-a09f-83037562c511\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:06.578988 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.578942 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d37144-0815-441e-a09f-83037562c511-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" (UID: \"e9d37144-0815-441e-a09f-83037562c511\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:06.579177 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.579028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55rd7\" (UniqueName: \"kubernetes.io/projected/e9d37144-0815-441e-a09f-83037562c511-kube-api-access-55rd7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" (UID: \"e9d37144-0815-441e-a09f-83037562c511\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:06.579423 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.579401 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d37144-0815-441e-a09f-83037562c511-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" (UID: \"e9d37144-0815-441e-a09f-83037562c511\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:06.589218 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.589174 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55rd7\" (UniqueName: \"kubernetes.io/projected/e9d37144-0815-441e-a09f-83037562c511-kube-api-access-55rd7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" (UID: \"e9d37144-0815-441e-a09f-83037562c511\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:06.712945 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.712893 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:06.851860 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:06.851775 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv"] Jun 09 03:10:06.855357 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:10:06.855310 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d37144_0815_441e_a09f_83037562c511.slice/crio-b63d8fe5d9895d5c5560b40643bf915dcfbd202ad8269b9010abdc49467d9af7 WatchSource:0}: Error finding container b63d8fe5d9895d5c5560b40643bf915dcfbd202ad8269b9010abdc49467d9af7: Status 404 returned error can't find the container with id b63d8fe5d9895d5c5560b40643bf915dcfbd202ad8269b9010abdc49467d9af7 Jun 09 03:10:07.862683 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:07.862631 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" event={"ID":"e9d37144-0815-441e-a09f-83037562c511","Type":"ContainerStarted","Data":"b63d8fe5d9895d5c5560b40643bf915dcfbd202ad8269b9010abdc49467d9af7"} Jun 09 03:10:08.817761 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:08.817730 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j5d52" Jun 09 03:10:11.833754 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:11.833675 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:14.900143 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:14.900101 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" event={"ID":"e9d37144-0815-441e-a09f-83037562c511","Type":"ContainerStarted","Data":"c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81"} Jun 09 03:10:14.900594 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:14.900249 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:14.924887 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:14.919469 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" podStartSLOduration=1.83525885 podStartE2EDuration="8.919434088s" totalImagesPullingTime="7.084175238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:10:14.918650165 +0000 UTC m=+648.828164513" watchObservedRunningTime="2026-06-09 03:10:14.919434088 +0000 UTC m=+648.828948432" Jun 09 03:10:25.907923 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:25.907884 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:26.803415 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.803326 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw"] Jun 09 03:10:26.807773 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.807756 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:26.826172 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.826143 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw"] Jun 09 03:10:26.863060 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.863001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5mk6\" (UniqueName: \"kubernetes.io/projected/f656d9ab-6621-4000-82e9-cd6f92d97155-kube-api-access-t5mk6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" (UID: \"f656d9ab-6621-4000-82e9-cd6f92d97155\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:26.863263 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.863083 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f656d9ab-6621-4000-82e9-cd6f92d97155-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" (UID: \"f656d9ab-6621-4000-82e9-cd6f92d97155\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:26.964499 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.964462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5mk6\" (UniqueName: \"kubernetes.io/projected/f656d9ab-6621-4000-82e9-cd6f92d97155-kube-api-access-t5mk6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" (UID: \"f656d9ab-6621-4000-82e9-cd6f92d97155\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:26.964928 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.964516 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f656d9ab-6621-4000-82e9-cd6f92d97155-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" (UID: \"f656d9ab-6621-4000-82e9-cd6f92d97155\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:26.964928 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.964853 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f656d9ab-6621-4000-82e9-cd6f92d97155-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" (UID: \"f656d9ab-6621-4000-82e9-cd6f92d97155\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:26.972145 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:26.972119 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5mk6\" (UniqueName: \"kubernetes.io/projected/f656d9ab-6621-4000-82e9-cd6f92d97155-kube-api-access-t5mk6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" (UID: \"f656d9ab-6621-4000-82e9-cd6f92d97155\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:27.117586 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.117483 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:27.259905 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.259812 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw"] Jun 09 03:10:27.262390 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:10:27.262361 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf656d9ab_6621_4000_82e9_cd6f92d97155.slice/crio-8b555a62cf758b599420b8de2094fe4c283289994ddfabe315b49efca268a363 WatchSource:0}: Error finding container 8b555a62cf758b599420b8de2094fe4c283289994ddfabe315b49efca268a363: Status 404 returned error can't find the container with id 8b555a62cf758b599420b8de2094fe4c283289994ddfabe315b49efca268a363 Jun 09 03:10:27.613224 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.613177 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw"] Jun 09 03:10:27.633688 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.633653 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw"] Jun 09 03:10:27.652219 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.652172 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv"] Jun 09 03:10:27.652417 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.652394 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" podUID="e9d37144-0815-441e-a09f-83037562c511" containerName="manager" containerID="cri-o://c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81" gracePeriod=2 Jun 09 03:10:27.665401 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.665366 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp"] Jun 09 03:10:27.665890 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.665866 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" containerName="manager" Jun 09 03:10:27.665890 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.665889 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" containerName="manager" Jun 09 03:10:27.666067 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.665984 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" containerName="manager" Jun 09 03:10:27.667076 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.667042 2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e9d37144-0815-441e-a09f-83037562c511\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-s74hv\": pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.669035 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.669015 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv"] Jun 09 03:10:27.669123 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.669101 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:27.675040 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.675017 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg"] Jun 09 03:10:27.675222 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.675188 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" containerName="manager" containerID="cri-o://86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1" gracePeriod=2 Jun 09 03:10:27.688696 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.688599 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26"] Jun 09 03:10:27.689070 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.689049 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d37144-0815-441e-a09f-83037562c511" containerName="manager" Jun 09 03:10:27.689070 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.689070 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d37144-0815-441e-a09f-83037562c511" containerName="manager" Jun 09 03:10:27.689270 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.689132 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d37144-0815-441e-a09f-83037562c511" containerName="manager" Jun 09 03:10:27.692323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.692298 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp"] Jun 09 03:10:27.692446 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.692394 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:27.692814 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.692790 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg"] Jun 09 03:10:27.708430 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.708399 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26"] Jun 09 03:10:27.715878 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.715833 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.717375 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.717347 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.717885 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.717865 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg"] Jun 09 03:10:27.718210 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.718187 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" containerName="manager" Jun 09 03:10:27.718246 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.718215 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" containerName="manager" Jun 09 03:10:27.718299 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.718290 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" containerName="manager" Jun 09 03:10:27.721244 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.721226 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" Jun 09 03:10:27.730717 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.730686 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg"] Jun 09 03:10:27.753143 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.753107 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.755992 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.755944 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.773076 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.773049 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk9pr\" (UniqueName: \"kubernetes.io/projected/c42662c6-e708-4935-b775-3f43ed4497bb-kube-api-access-mk9pr\") pod \"limitador-operator-controller-manager-859d7b78bd-8sbzg\" (UID: \"c42662c6-e708-4935-b775-3f43ed4497bb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" Jun 09 03:10:27.773276 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.773091 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f68db\" (UniqueName: \"kubernetes.io/projected/fcea49b8-06b9-4cf3-91a3-1633c30f6191-kube-api-access-f68db\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xxtgp\" (UID: \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:27.773276 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.773163 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcea49b8-06b9-4cf3-91a3-1633c30f6191-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xxtgp\" (UID: \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:27.773276 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.773215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slkfm\" (UniqueName: \"kubernetes.io/projected/3003f75f-a822-48be-9e7a-93997f622e4d-kube-api-access-slkfm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rbd26\" (UID: \"3003f75f-a822-48be-9e7a-93997f622e4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:27.773276 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.773253 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3003f75f-a822-48be-9e7a-93997f622e4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rbd26\" (UID: \"3003f75f-a822-48be-9e7a-93997f622e4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:27.776456 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.776425 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.874148 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.874057 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68db\" (UniqueName: \"kubernetes.io/projected/fcea49b8-06b9-4cf3-91a3-1633c30f6191-kube-api-access-f68db\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xxtgp\" (UID: \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:27.874148 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.874120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcea49b8-06b9-4cf3-91a3-1633c30f6191-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xxtgp\" (UID: \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:27.874396 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.874147 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slkfm\" (UniqueName: \"kubernetes.io/projected/3003f75f-a822-48be-9e7a-93997f622e4d-kube-api-access-slkfm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rbd26\" (UID: \"3003f75f-a822-48be-9e7a-93997f622e4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:27.874396 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.874184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3003f75f-a822-48be-9e7a-93997f622e4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rbd26\" (UID: \"3003f75f-a822-48be-9e7a-93997f622e4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:27.874396 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.874265 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk9pr\" (UniqueName: \"kubernetes.io/projected/c42662c6-e708-4935-b775-3f43ed4497bb-kube-api-access-mk9pr\") pod \"limitador-operator-controller-manager-859d7b78bd-8sbzg\" (UID: \"c42662c6-e708-4935-b775-3f43ed4497bb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" Jun 09 03:10:27.874585 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.874559 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcea49b8-06b9-4cf3-91a3-1633c30f6191-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xxtgp\" (UID: \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:27.874653 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.874636 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3003f75f-a822-48be-9e7a-93997f622e4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rbd26\" (UID: \"3003f75f-a822-48be-9e7a-93997f622e4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:27.882796 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.882759 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slkfm\" (UniqueName: \"kubernetes.io/projected/3003f75f-a822-48be-9e7a-93997f622e4d-kube-api-access-slkfm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rbd26\" (UID: \"3003f75f-a822-48be-9e7a-93997f622e4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:27.882970 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.882924 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk9pr\" (UniqueName: \"kubernetes.io/projected/c42662c6-e708-4935-b775-3f43ed4497bb-kube-api-access-mk9pr\") pod \"limitador-operator-controller-manager-859d7b78bd-8sbzg\" (UID: \"c42662c6-e708-4935-b775-3f43ed4497bb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" Jun 09 03:10:27.883239 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.883218 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68db\" (UniqueName: \"kubernetes.io/projected/fcea49b8-06b9-4cf3-91a3-1633c30f6191-kube-api-access-f68db\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xxtgp\" (UID: \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:27.907299 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.907270 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.909135 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.909111 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.910484 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.910463 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.914476 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.914456 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.915974 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.915951 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.917223 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.917187 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.960839 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.960805 2564 generic.go:356] "Generic (PLEG): container finished" podID="e9d37144-0815-441e-a09f-83037562c511" containerID="c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81" exitCode=0 Jun 09 03:10:27.961033 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.960852 2564 scope.go:117] "RemoveContainer" containerID="c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81" Jun 09 03:10:27.961033 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.960856 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.962815 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.962770 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.963124 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.963099 2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" container="manager" Jun 09 03:10:27.964057 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.964025 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.964864 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.964842 2564 generic.go:356] "Generic (PLEG): container finished" podID="596169dd-de1f-445a-8e81-4c3d0670d6cf" containerID="86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1" exitCode=0 Jun 09 03:10:27.965270 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.964873 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.965337 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.965259 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.966496 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.966468 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:27.967588 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.967560 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.968800 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.968773 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:27.969948 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.969926 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:27.971064 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.971042 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:27.971362 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.971343 2564 scope.go:117] "RemoveContainer" containerID="c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81" Jun 09 03:10:27.971684 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:27.971660 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81\": container with ID starting with c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81 not found: ID does not exist" containerID="c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81" Jun 09 03:10:27.971761 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.971696 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81"} err="failed to get container status \"c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81\": rpc error: code = NotFound desc = could not find container \"c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81\": container with ID starting with c5b0df196eb8b71d0b29c06991bbfcd56172c33fa492fafea2cfe09973db9d81 not found: ID does not exist" Jun 09 03:10:27.971761 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.971720 2564 scope.go:117] "RemoveContainer" containerID="86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1" Jun 09 03:10:27.975116 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.975091 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55rd7\" (UniqueName: \"kubernetes.io/projected/e9d37144-0815-441e-a09f-83037562c511-kube-api-access-55rd7\") pod \"e9d37144-0815-441e-a09f-83037562c511\" (UID: \"e9d37144-0815-441e-a09f-83037562c511\") " Jun 09 03:10:27.975228 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.975165 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vhgb\" (UniqueName: \"kubernetes.io/projected/596169dd-de1f-445a-8e81-4c3d0670d6cf-kube-api-access-9vhgb\") pod \"596169dd-de1f-445a-8e81-4c3d0670d6cf\" (UID: \"596169dd-de1f-445a-8e81-4c3d0670d6cf\") " Jun 09 03:10:27.975228 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.975221 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d37144-0815-441e-a09f-83037562c511-extensions-socket-volume\") pod \"e9d37144-0815-441e-a09f-83037562c511\" (UID: \"e9d37144-0815-441e-a09f-83037562c511\") " Jun 09 03:10:27.975855 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.975828 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9d37144-0815-441e-a09f-83037562c511-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e9d37144-0815-441e-a09f-83037562c511" (UID: "e9d37144-0815-441e-a09f-83037562c511"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:10:27.977173 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.977145 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596169dd-de1f-445a-8e81-4c3d0670d6cf-kube-api-access-9vhgb" (OuterVolumeSpecName: "kube-api-access-9vhgb") pod "596169dd-de1f-445a-8e81-4c3d0670d6cf" (UID: "596169dd-de1f-445a-8e81-4c3d0670d6cf"). InnerVolumeSpecName "kube-api-access-9vhgb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:10:27.977277 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:27.977157 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9d37144-0815-441e-a09f-83037562c511-kube-api-access-55rd7" (OuterVolumeSpecName: "kube-api-access-55rd7") pod "e9d37144-0815-441e-a09f-83037562c511" (UID: "e9d37144-0815-441e-a09f-83037562c511"). InnerVolumeSpecName "kube-api-access-55rd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:10:28.000541 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.000521 2564 scope.go:117] "RemoveContainer" containerID="86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1" Jun 09 03:10:28.000887 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:28.000866 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1\": container with ID starting with 86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1 not found: ID does not exist" containerID="86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1" Jun 09 03:10:28.000959 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.000895 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1"} err="failed to get container status \"86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1\": rpc error: code = NotFound desc = could not find container \"86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1\": container with ID starting with 86bf2857d7a3b418aefb80d26cb511cabbb43ced52280d47a14797c5f90424d1 not found: ID does not exist" Jun 09 03:10:28.076318 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.076281 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55rd7\" (UniqueName: \"kubernetes.io/projected/e9d37144-0815-441e-a09f-83037562c511-kube-api-access-55rd7\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:28.076318 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.076315 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vhgb\" (UniqueName: \"kubernetes.io/projected/596169dd-de1f-445a-8e81-4c3d0670d6cf-kube-api-access-9vhgb\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:28.076318 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.076326 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d37144-0815-441e-a09f-83037562c511-extensions-socket-volume\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:28.088172 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.088133 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:28.097110 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.097078 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:28.103299 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.103266 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" Jun 09 03:10:28.255066 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.254979 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26"] Jun 09 03:10:28.258673 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:10:28.258629 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3003f75f_a822_48be_9e7a_93997f622e4d.slice/crio-fbe502512394f6d9f30e7992c5a146b8ac558dbee600d6f01d97d7d113557bd0 WatchSource:0}: Error finding container fbe502512394f6d9f30e7992c5a146b8ac558dbee600d6f01d97d7d113557bd0: Status 404 returned error can't find the container with id fbe502512394f6d9f30e7992c5a146b8ac558dbee600d6f01d97d7d113557bd0 Jun 09 03:10:28.274809 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:28.274773 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:28.276021 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:28.275982 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:28.276739 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.276712 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp"] Jun 09 03:10:28.277466 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:28.277289 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:28.279645 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:10:28.279618 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcea49b8_06b9_4cf3_91a3_1633c30f6191.slice/crio-a65dc6b3e90749d2969cc5d092a4230e88d01a94ad6b7ea2d213ec1185d8641a WatchSource:0}: Error finding container a65dc6b3e90749d2969cc5d092a4230e88d01a94ad6b7ea2d213ec1185d8641a: Status 404 returned error can't find the container with id a65dc6b3e90749d2969cc5d092a4230e88d01a94ad6b7ea2d213ec1185d8641a Jun 09 03:10:28.279894 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:28.279862 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s74hv\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="e9d37144-0815-441e-a09f-83037562c511" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s74hv" Jun 09 03:10:28.280991 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:28.280963 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:28.282284 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:28.282234 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cngbg\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cngbg" Jun 09 03:10:28.313170 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.313143 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg"] Jun 09 03:10:28.324598 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:10:28.324565 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42662c6_e708_4935_b775_3f43ed4497bb.slice/crio-43253c3ea71c2239f228733d7f9eeeb0fe6a27af5bd927b8b2d032e2c1766b94 WatchSource:0}: Error finding container 43253c3ea71c2239f228733d7f9eeeb0fe6a27af5bd927b8b2d032e2c1766b94: Status 404 returned error can't find the container with id 43253c3ea71c2239f228733d7f9eeeb0fe6a27af5bd927b8b2d032e2c1766b94 Jun 09 03:10:28.551954 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.551913 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="596169dd-de1f-445a-8e81-4c3d0670d6cf" path="/var/lib/kubelet/pods/596169dd-de1f-445a-8e81-4c3d0670d6cf/volumes" Jun 09 03:10:28.552413 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.552396 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9d37144-0815-441e-a09f-83037562c511" path="/var/lib/kubelet/pods/e9d37144-0815-441e-a09f-83037562c511/volumes" Jun 09 03:10:28.972891 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.972848 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" event={"ID":"3003f75f-a822-48be-9e7a-93997f622e4d","Type":"ContainerStarted","Data":"3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2"} Jun 09 03:10:28.973354 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.972911 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" event={"ID":"3003f75f-a822-48be-9e7a-93997f622e4d","Type":"ContainerStarted","Data":"fbe502512394f6d9f30e7992c5a146b8ac558dbee600d6f01d97d7d113557bd0"} Jun 09 03:10:28.973354 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.972951 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:28.974680 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.974650 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" event={"ID":"fcea49b8-06b9-4cf3-91a3-1633c30f6191","Type":"ContainerStarted","Data":"62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57"} Jun 09 03:10:28.974804 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.974693 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" event={"ID":"fcea49b8-06b9-4cf3-91a3-1633c30f6191","Type":"ContainerStarted","Data":"a65dc6b3e90749d2969cc5d092a4230e88d01a94ad6b7ea2d213ec1185d8641a"} Jun 09 03:10:28.974804 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.974715 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:28.977252 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.977223 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" containerName="manager" containerID="cri-o://0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904" gracePeriod=2 Jun 09 03:10:28.978886 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.978858 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" event={"ID":"c42662c6-e708-4935-b775-3f43ed4497bb","Type":"ContainerStarted","Data":"11554fe45934c4611e5c83956de7b74c885e63fb9f95a65b3b71e4b7a383c6fc"} Jun 09 03:10:28.979007 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.978910 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" event={"ID":"c42662c6-e708-4935-b775-3f43ed4497bb","Type":"ContainerStarted","Data":"43253c3ea71c2239f228733d7f9eeeb0fe6a27af5bd927b8b2d032e2c1766b94"} Jun 09 03:10:28.979007 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.978954 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" Jun 09 03:10:28.992600 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:28.992551 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" podStartSLOduration=1.9925344379999999 podStartE2EDuration="1.992534438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:10:28.991614718 +0000 UTC m=+662.901129064" watchObservedRunningTime="2026-06-09 03:10:28.992534438 +0000 UTC m=+662.902048784" Jun 09 03:10:29.007881 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.007730 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" podStartSLOduration=2.007711455 podStartE2EDuration="2.007711455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:10:29.007635345 +0000 UTC m=+662.917149694" watchObservedRunningTime="2026-06-09 03:10:29.007711455 +0000 UTC m=+662.917225805" Jun 09 03:10:29.025003 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.024947 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" podStartSLOduration=2.024928 podStartE2EDuration="2.024928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:10:29.023466963 +0000 UTC m=+662.932981311" watchObservedRunningTime="2026-06-09 03:10:29.024928 +0000 UTC m=+662.934442347" Jun 09 03:10:29.229974 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.229897 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:29.232041 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:29.232009 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:29.286537 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.286494 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f656d9ab-6621-4000-82e9-cd6f92d97155-extensions-socket-volume\") pod \"f656d9ab-6621-4000-82e9-cd6f92d97155\" (UID: \"f656d9ab-6621-4000-82e9-cd6f92d97155\") " Jun 09 03:10:29.286713 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.286571 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5mk6\" (UniqueName: \"kubernetes.io/projected/f656d9ab-6621-4000-82e9-cd6f92d97155-kube-api-access-t5mk6\") pod \"f656d9ab-6621-4000-82e9-cd6f92d97155\" (UID: \"f656d9ab-6621-4000-82e9-cd6f92d97155\") " Jun 09 03:10:29.286915 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.286896 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f656d9ab-6621-4000-82e9-cd6f92d97155-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f656d9ab-6621-4000-82e9-cd6f92d97155" (UID: "f656d9ab-6621-4000-82e9-cd6f92d97155"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:10:29.288644 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.288623 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f656d9ab-6621-4000-82e9-cd6f92d97155-kube-api-access-t5mk6" (OuterVolumeSpecName: "kube-api-access-t5mk6") pod "f656d9ab-6621-4000-82e9-cd6f92d97155" (UID: "f656d9ab-6621-4000-82e9-cd6f92d97155"). InnerVolumeSpecName "kube-api-access-t5mk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:10:29.387872 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.387834 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f656d9ab-6621-4000-82e9-cd6f92d97155-extensions-socket-volume\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:29.387872 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.387866 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5mk6\" (UniqueName: \"kubernetes.io/projected/f656d9ab-6621-4000-82e9-cd6f92d97155-kube-api-access-t5mk6\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:29.986842 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.986803 2564 generic.go:356] "Generic (PLEG): container finished" podID="f656d9ab-6621-4000-82e9-cd6f92d97155" containerID="0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904" exitCode=0 Jun 09 03:10:29.986842 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.986846 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:29.987403 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.986907 2564 scope.go:117] "RemoveContainer" containerID="0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904" Jun 09 03:10:29.988814 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:29.988791 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:29.995991 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.995972 2564 scope.go:117] "RemoveContainer" containerID="0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904" Jun 09 03:10:29.996280 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:29.996261 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904\": container with ID starting with 0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904 not found: ID does not exist" containerID="0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904" Jun 09 03:10:29.996355 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:29.996294 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904"} err="failed to get container status \"0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904\": rpc error: code = NotFound desc = could not find container \"0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904\": container with ID starting with 0a4f5a8dad025f0c8c3861aa9b1f02470beeab94eceeaa1ac9dab6f0a6680904 not found: ID does not exist" Jun 09 03:10:29.996533 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:29.996515 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6x8cw\" is forbidden: User \"system:node:ip-10-0-130-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-143.ec2.internal' and this object" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6x8cw" Jun 09 03:10:30.551323 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:30.551288 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f656d9ab-6621-4000-82e9-cd6f92d97155" path="/var/lib/kubelet/pods/f656d9ab-6621-4000-82e9-cd6f92d97155/volumes" Jun 09 03:10:39.989884 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:39.989851 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:39.990433 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:39.989928 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8sbzg" Jun 09 03:10:39.990433 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:39.990191 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:40.067630 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.067600 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26"] Jun 09 03:10:40.067842 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.067811 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" podUID="3003f75f-a822-48be-9e7a-93997f622e4d" containerName="manager" containerID="cri-o://3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2" gracePeriod=10 Jun 09 03:10:40.301860 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.301836 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl"] Jun 09 03:10:40.305815 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.305797 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:40.311374 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.311356 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:40.340839 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.340815 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl"] Jun 09 03:10:40.381786 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.381761 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slkfm\" (UniqueName: \"kubernetes.io/projected/3003f75f-a822-48be-9e7a-93997f622e4d-kube-api-access-slkfm\") pod \"3003f75f-a822-48be-9e7a-93997f622e4d\" (UID: \"3003f75f-a822-48be-9e7a-93997f622e4d\") " Jun 09 03:10:40.381873 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.381794 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3003f75f-a822-48be-9e7a-93997f622e4d-extensions-socket-volume\") pod \"3003f75f-a822-48be-9e7a-93997f622e4d\" (UID: \"3003f75f-a822-48be-9e7a-93997f622e4d\") " Jun 09 03:10:40.382021 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.382001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j5bgl\" (UID: \"7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:40.382059 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.382040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shxf2\" (UniqueName: \"kubernetes.io/projected/7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb-kube-api-access-shxf2\") pod \"kuadrant-operator-controller-manager-86878c769f-j5bgl\" (UID: \"7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:40.382269 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.382238 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3003f75f-a822-48be-9e7a-93997f622e4d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3003f75f-a822-48be-9e7a-93997f622e4d" (UID: "3003f75f-a822-48be-9e7a-93997f622e4d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:10:40.383710 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.383690 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3003f75f-a822-48be-9e7a-93997f622e4d-kube-api-access-slkfm" (OuterVolumeSpecName: "kube-api-access-slkfm") pod "3003f75f-a822-48be-9e7a-93997f622e4d" (UID: "3003f75f-a822-48be-9e7a-93997f622e4d"). InnerVolumeSpecName "kube-api-access-slkfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:10:40.483047 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.483017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j5bgl\" (UID: \"7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:40.483047 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.483048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shxf2\" (UniqueName: \"kubernetes.io/projected/7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb-kube-api-access-shxf2\") pod \"kuadrant-operator-controller-manager-86878c769f-j5bgl\" (UID: \"7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:40.483289 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.483107 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slkfm\" (UniqueName: \"kubernetes.io/projected/3003f75f-a822-48be-9e7a-93997f622e4d-kube-api-access-slkfm\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:40.483289 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.483116 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3003f75f-a822-48be-9e7a-93997f622e4d-extensions-socket-volume\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:40.483386 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.483368 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j5bgl\" (UID: \"7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:40.490120 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.490090 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shxf2\" (UniqueName: \"kubernetes.io/projected/7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb-kube-api-access-shxf2\") pod \"kuadrant-operator-controller-manager-86878c769f-j5bgl\" (UID: \"7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:40.616924 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.616877 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:40.746460 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:40.746435 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl"] Jun 09 03:10:40.749041 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:10:40.749018 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7867e3d0_6e5a_4d19_abbe_7dd6b30a3eeb.slice/crio-34b2b277acd5bb6e4b5ccfa418d7e23e824ac5d372f2ac48f4458d48cd8674e0 WatchSource:0}: Error finding container 34b2b277acd5bb6e4b5ccfa418d7e23e824ac5d372f2ac48f4458d48cd8674e0: Status 404 returned error can't find the container with id 34b2b277acd5bb6e4b5ccfa418d7e23e824ac5d372f2ac48f4458d48cd8674e0 Jun 09 03:10:41.045147 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.045117 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" event={"ID":"7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb","Type":"ContainerStarted","Data":"d6537fdb77978d9c695e50248e8cf2ab8e50c9bbc7d6ff0d2fb66eb4cff2990d"} Jun 09 03:10:41.045582 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.045175 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" event={"ID":"7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb","Type":"ContainerStarted","Data":"34b2b277acd5bb6e4b5ccfa418d7e23e824ac5d372f2ac48f4458d48cd8674e0"} Jun 09 03:10:41.045582 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.045218 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:41.046475 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.046453 2564 generic.go:356] "Generic (PLEG): container finished" podID="3003f75f-a822-48be-9e7a-93997f622e4d" containerID="3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2" exitCode=0 Jun 09 03:10:41.046538 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.046485 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" event={"ID":"3003f75f-a822-48be-9e7a-93997f622e4d","Type":"ContainerDied","Data":"3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2"} Jun 09 03:10:41.046538 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.046515 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" event={"ID":"3003f75f-a822-48be-9e7a-93997f622e4d","Type":"ContainerDied","Data":"fbe502512394f6d9f30e7992c5a146b8ac558dbee600d6f01d97d7d113557bd0"} Jun 09 03:10:41.046538 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.046534 2564 scope.go:117] "RemoveContainer" containerID="3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2" Jun 09 03:10:41.046631 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.046486 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26" Jun 09 03:10:41.054942 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.054924 2564 scope.go:117] "RemoveContainer" containerID="3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2" Jun 09 03:10:41.055191 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:41.055174 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2\": container with ID starting with 3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2 not found: ID does not exist" containerID="3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2" Jun 09 03:10:41.055253 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.055214 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2"} err="failed to get container status \"3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2\": rpc error: code = NotFound desc = could not find container \"3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2\": container with ID starting with 3615404a6f42df2fbce2d012f3ccc777a157dca945e2e44fecdb26e6d73685c2 not found: ID does not exist" Jun 09 03:10:41.061447 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.061406 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" podStartSLOduration=1.061383396 podStartE2EDuration="1.061383396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:10:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:10:41.060788205 +0000 UTC m=+674.970302549" watchObservedRunningTime="2026-06-09 03:10:41.061383396 +0000 UTC m=+674.970897745" Jun 09 03:10:41.074165 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.074145 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26"] Jun 09 03:10:41.079904 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:41.079879 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rbd26"] Jun 09 03:10:42.550711 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:42.550683 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3003f75f-a822-48be-9e7a-93997f622e4d" path="/var/lib/kubelet/pods/3003f75f-a822-48be-9e7a-93997f622e4d/volumes" Jun 09 03:10:52.054936 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.054901 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j5bgl" Jun 09 03:10:52.098868 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.098835 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp"] Jun 09 03:10:52.099077 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.099056 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" podUID="fcea49b8-06b9-4cf3-91a3-1633c30f6191" containerName="manager" containerID="cri-o://62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57" gracePeriod=10 Jun 09 03:10:52.353227 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.353175 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:52.489984 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.489948 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f68db\" (UniqueName: \"kubernetes.io/projected/fcea49b8-06b9-4cf3-91a3-1633c30f6191-kube-api-access-f68db\") pod \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\" (UID: \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\") " Jun 09 03:10:52.490163 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.490000 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcea49b8-06b9-4cf3-91a3-1633c30f6191-extensions-socket-volume\") pod \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\" (UID: \"fcea49b8-06b9-4cf3-91a3-1633c30f6191\") " Jun 09 03:10:52.490446 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.490420 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcea49b8-06b9-4cf3-91a3-1633c30f6191-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fcea49b8-06b9-4cf3-91a3-1633c30f6191" (UID: "fcea49b8-06b9-4cf3-91a3-1633c30f6191"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 03:10:52.492183 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.492160 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcea49b8-06b9-4cf3-91a3-1633c30f6191-kube-api-access-f68db" (OuterVolumeSpecName: "kube-api-access-f68db") pod "fcea49b8-06b9-4cf3-91a3-1633c30f6191" (UID: "fcea49b8-06b9-4cf3-91a3-1633c30f6191"). InnerVolumeSpecName "kube-api-access-f68db". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:10:52.591455 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.591367 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f68db\" (UniqueName: \"kubernetes.io/projected/fcea49b8-06b9-4cf3-91a3-1633c30f6191-kube-api-access-f68db\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:52.591455 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:52.591396 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcea49b8-06b9-4cf3-91a3-1633c30f6191-extensions-socket-volume\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:10:53.106948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.106902 2564 generic.go:356] "Generic (PLEG): container finished" podID="fcea49b8-06b9-4cf3-91a3-1633c30f6191" containerID="62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57" exitCode=0 Jun 09 03:10:53.106948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.106943 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" event={"ID":"fcea49b8-06b9-4cf3-91a3-1633c30f6191","Type":"ContainerDied","Data":"62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57"} Jun 09 03:10:53.107451 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.106997 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" event={"ID":"fcea49b8-06b9-4cf3-91a3-1633c30f6191","Type":"ContainerDied","Data":"a65dc6b3e90749d2969cc5d092a4230e88d01a94ad6b7ea2d213ec1185d8641a"} Jun 09 03:10:53.107451 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.106954 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp" Jun 09 03:10:53.107451 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.107019 2564 scope.go:117] "RemoveContainer" containerID="62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57" Jun 09 03:10:53.116399 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.116373 2564 scope.go:117] "RemoveContainer" containerID="62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57" Jun 09 03:10:53.116662 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:10:53.116644 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57\": container with ID starting with 62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57 not found: ID does not exist" containerID="62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57" Jun 09 03:10:53.116716 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.116670 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57"} err="failed to get container status \"62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57\": rpc error: code = NotFound desc = could not find container \"62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57\": container with ID starting with 62e7916766ab5679ce287c75322871982fb4f48060a9497e9d64661cd23a7f57 not found: ID does not exist" Jun 09 03:10:53.121726 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.121700 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp"] Jun 09 03:10:53.126558 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:53.126537 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xxtgp"] Jun 09 03:10:54.550793 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:54.550762 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcea49b8-06b9-4cf3-91a3-1633c30f6191" path="/var/lib/kubelet/pods/fcea49b8-06b9-4cf3-91a3-1633c30f6191/volumes" Jun 09 03:10:56.367321 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.367285 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw"] Jun 09 03:10:56.367830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.367646 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3003f75f-a822-48be-9e7a-93997f622e4d" containerName="manager" Jun 09 03:10:56.367830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.367657 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3003f75f-a822-48be-9e7a-93997f622e4d" containerName="manager" Jun 09 03:10:56.367830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.367670 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcea49b8-06b9-4cf3-91a3-1633c30f6191" containerName="manager" Jun 09 03:10:56.367830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.367676 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcea49b8-06b9-4cf3-91a3-1633c30f6191" containerName="manager" Jun 09 03:10:56.367830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.367735 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3003f75f-a822-48be-9e7a-93997f622e4d" containerName="manager" Jun 09 03:10:56.367830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.367743 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcea49b8-06b9-4cf3-91a3-1633c30f6191" containerName="manager" Jun 09 03:10:56.370853 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.370823 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.372824 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.372802 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mzx72\"" Jun 09 03:10:56.381872 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.381839 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw"] Jun 09 03:10:56.527119 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.527119 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.527370 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.527370 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527234 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.527370 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.527370 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527318 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.527511 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527371 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd782738-6218-4c10-ac8b-fbe789ae065e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.527511 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527389 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ptjf\" (UniqueName: \"kubernetes.io/projected/cd782738-6218-4c10-ac8b-fbe789ae065e-kube-api-access-4ptjf\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.527511 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.527453 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.628643 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628542 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.628643 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628591 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.628643 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628612 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.628908 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628729 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.628908 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628788 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.628908 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628826 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.629074 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628914 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd782738-6218-4c10-ac8b-fbe789ae065e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.629074 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628938 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ptjf\" (UniqueName: \"kubernetes.io/projected/cd782738-6218-4c10-ac8b-fbe789ae065e-kube-api-access-4ptjf\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.629074 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628943 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.629074 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.629074 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.628997 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.629074 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.629034 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.629423 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.629240 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.629592 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.629571 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd782738-6218-4c10-ac8b-fbe789ae065e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.631345 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.631321 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.631520 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.631499 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.636065 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.636030 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd782738-6218-4c10-ac8b-fbe789ae065e-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.636269 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.636240 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ptjf\" (UniqueName: \"kubernetes.io/projected/cd782738-6218-4c10-ac8b-fbe789ae065e-kube-api-access-4ptjf\") pod \"maas-default-gateway-openshift-default-687ff6996-kjtgw\" (UID: \"cd782738-6218-4c10-ac8b-fbe789ae065e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.683020 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.682988 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:56.856234 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.855167 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw"] Jun 09 03:10:56.869605 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:10:56.869565 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd782738_6218_4c10_ac8b_fbe789ae065e.slice/crio-57155643cacf5cee33f58370f1fe06c2c7ce5573cab71f2511844eb8f49dca92 WatchSource:0}: Error finding container 57155643cacf5cee33f58370f1fe06c2c7ce5573cab71f2511844eb8f49dca92: Status 404 returned error can't find the container with id 57155643cacf5cee33f58370f1fe06c2c7ce5573cab71f2511844eb8f49dca92 Jun 09 03:10:56.872094 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.872043 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:10:56.872239 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.872138 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:10:56.872239 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:56.872186 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:10:57.131132 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:57.131095 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" event={"ID":"cd782738-6218-4c10-ac8b-fbe789ae065e","Type":"ContainerStarted","Data":"6b7bee553bfbad89e978300e15cf77128fadaba7c3f29d94a38727e3c06fa493"} Jun 09 03:10:57.131332 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:57.131150 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" event={"ID":"cd782738-6218-4c10-ac8b-fbe789ae065e","Type":"ContainerStarted","Data":"57155643cacf5cee33f58370f1fe06c2c7ce5573cab71f2511844eb8f49dca92"} Jun 09 03:10:57.147689 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:57.147543 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" podStartSLOduration=1.147521272 podStartE2EDuration="1.147521272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:10:57.146788492 +0000 UTC m=+691.056302839" watchObservedRunningTime="2026-06-09 03:10:57.147521272 +0000 UTC m=+691.057035614" Jun 09 03:10:57.684054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:57.684011 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:57.689439 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:57.689413 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:58.135946 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:58.135911 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:10:58.136822 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:10:58.136805 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:11:12.661494 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.661461 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:11:12.664535 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.664508 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:12.666218 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.666184 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 09 03:11:12.666351 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.666245 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j572w\"" Jun 09 03:11:12.671668 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.671638 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:11:12.679126 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.679103 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjx45\" (UniqueName: \"kubernetes.io/projected/da0d94b3-702e-4fb9-8de6-245b4a23909a-kube-api-access-zjx45\") pod \"limitador-limitador-54d6d988d5-4gjhs\" (UID: \"da0d94b3-702e-4fb9-8de6-245b4a23909a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:12.679252 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.679169 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da0d94b3-702e-4fb9-8de6-245b4a23909a-config-file\") pod \"limitador-limitador-54d6d988d5-4gjhs\" (UID: \"da0d94b3-702e-4fb9-8de6-245b4a23909a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:12.701443 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.701404 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:11:12.779790 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.779744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjx45\" (UniqueName: \"kubernetes.io/projected/da0d94b3-702e-4fb9-8de6-245b4a23909a-kube-api-access-zjx45\") pod \"limitador-limitador-54d6d988d5-4gjhs\" (UID: \"da0d94b3-702e-4fb9-8de6-245b4a23909a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:12.779979 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.779815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da0d94b3-702e-4fb9-8de6-245b4a23909a-config-file\") pod \"limitador-limitador-54d6d988d5-4gjhs\" (UID: \"da0d94b3-702e-4fb9-8de6-245b4a23909a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:12.780550 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.780531 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da0d94b3-702e-4fb9-8de6-245b4a23909a-config-file\") pod \"limitador-limitador-54d6d988d5-4gjhs\" (UID: \"da0d94b3-702e-4fb9-8de6-245b4a23909a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:12.787318 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.787283 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjx45\" (UniqueName: \"kubernetes.io/projected/da0d94b3-702e-4fb9-8de6-245b4a23909a-kube-api-access-zjx45\") pod \"limitador-limitador-54d6d988d5-4gjhs\" (UID: \"da0d94b3-702e-4fb9-8de6-245b4a23909a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:12.977753 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:12.977715 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:13.112893 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:13.112861 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:11:13.115685 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:11:13.115656 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0d94b3_702e_4fb9_8de6_245b4a23909a.slice/crio-b7c05adeac4c0333608080335845c2903c01621f115f2c3da4c2f7c22dc26020 WatchSource:0}: Error finding container b7c05adeac4c0333608080335845c2903c01621f115f2c3da4c2f7c22dc26020: Status 404 returned error can't find the container with id b7c05adeac4c0333608080335845c2903c01621f115f2c3da4c2f7c22dc26020 Jun 09 03:11:13.212249 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:13.212183 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" event={"ID":"da0d94b3-702e-4fb9-8de6-245b4a23909a","Type":"ContainerStarted","Data":"b7c05adeac4c0333608080335845c2903c01621f115f2c3da4c2f7c22dc26020"} Jun 09 03:11:16.230930 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:16.230850 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" event={"ID":"da0d94b3-702e-4fb9-8de6-245b4a23909a","Type":"ContainerStarted","Data":"e38db53c7dd7b38c85ef29b3902692e265e243181b1c41e77f4d51caad24b72a"} Jun 09 03:11:16.231319 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:16.230986 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:11:16.244753 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:16.244384 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" podStartSLOduration=1.785803858 podStartE2EDuration="4.244364054s" totalImagesPullingTime="2.458560196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:11:16.24361822 +0000 UTC m=+710.153132559" watchObservedRunningTime="2026-06-09 03:11:16.244364054 +0000 UTC m=+710.153878402" Jun 09 03:11:27.237699 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:11:27.237670 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4gjhs" Jun 09 03:12:41.337810 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.337770 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-vz9sg"] Jun 09 03:12:41.340350 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.340326 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:41.342657 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.342625 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 03:12:41.342811 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.342732 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 03:12:41.343072 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.343057 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-42zsb\"" Jun 09 03:12:41.350695 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.350659 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-vz9sg"] Jun 09 03:12:41.439820 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.439775 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j522d\" (UniqueName: \"kubernetes.io/projected/a69d8535-cb27-4360-92e1-c481950bd98a-kube-api-access-j522d\") pod \"maas-controller-7bbdb77c66-vz9sg\" (UID: \"a69d8535-cb27-4360-92e1-c481950bd98a\") " pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:41.440008 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.439857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a69d8535-cb27-4360-92e1-c481950bd98a-webhook-cert\") pod \"maas-controller-7bbdb77c66-vz9sg\" (UID: \"a69d8535-cb27-4360-92e1-c481950bd98a\") " pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:41.517961 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.517914 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8c57d4bd7-9mmp9"] Jun 09 03:12:41.520688 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.520659 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:41.529661 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.529626 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8c57d4bd7-9mmp9"] Jun 09 03:12:41.540608 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.540560 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j522d\" (UniqueName: \"kubernetes.io/projected/a69d8535-cb27-4360-92e1-c481950bd98a-kube-api-access-j522d\") pod \"maas-controller-7bbdb77c66-vz9sg\" (UID: \"a69d8535-cb27-4360-92e1-c481950bd98a\") " pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:41.540823 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.540638 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a69d8535-cb27-4360-92e1-c481950bd98a-webhook-cert\") pod \"maas-controller-7bbdb77c66-vz9sg\" (UID: \"a69d8535-cb27-4360-92e1-c481950bd98a\") " pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:41.543382 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.543346 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a69d8535-cb27-4360-92e1-c481950bd98a-webhook-cert\") pod \"maas-controller-7bbdb77c66-vz9sg\" (UID: \"a69d8535-cb27-4360-92e1-c481950bd98a\") " pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:41.547619 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.547593 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j522d\" (UniqueName: \"kubernetes.io/projected/a69d8535-cb27-4360-92e1-c481950bd98a-kube-api-access-j522d\") pod \"maas-controller-7bbdb77c66-vz9sg\" (UID: \"a69d8535-cb27-4360-92e1-c481950bd98a\") " pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:41.641612 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.641516 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac742c0a-ca0a-42dd-b3e9-1afeff65be8c-webhook-cert\") pod \"maas-controller-8c57d4bd7-9mmp9\" (UID: \"ac742c0a-ca0a-42dd-b3e9-1afeff65be8c\") " pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:41.641612 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.641566 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctt8r\" (UniqueName: \"kubernetes.io/projected/ac742c0a-ca0a-42dd-b3e9-1afeff65be8c-kube-api-access-ctt8r\") pod \"maas-controller-8c57d4bd7-9mmp9\" (UID: \"ac742c0a-ca0a-42dd-b3e9-1afeff65be8c\") " pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:41.652092 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.652060 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:41.746065 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.743175 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac742c0a-ca0a-42dd-b3e9-1afeff65be8c-webhook-cert\") pod \"maas-controller-8c57d4bd7-9mmp9\" (UID: \"ac742c0a-ca0a-42dd-b3e9-1afeff65be8c\") " pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:41.746065 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.743288 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctt8r\" (UniqueName: \"kubernetes.io/projected/ac742c0a-ca0a-42dd-b3e9-1afeff65be8c-kube-api-access-ctt8r\") pod \"maas-controller-8c57d4bd7-9mmp9\" (UID: \"ac742c0a-ca0a-42dd-b3e9-1afeff65be8c\") " pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:41.755802 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.755768 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctt8r\" (UniqueName: \"kubernetes.io/projected/ac742c0a-ca0a-42dd-b3e9-1afeff65be8c-kube-api-access-ctt8r\") pod \"maas-controller-8c57d4bd7-9mmp9\" (UID: \"ac742c0a-ca0a-42dd-b3e9-1afeff65be8c\") " pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:41.755971 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.755810 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac742c0a-ca0a-42dd-b3e9-1afeff65be8c-webhook-cert\") pod \"maas-controller-8c57d4bd7-9mmp9\" (UID: \"ac742c0a-ca0a-42dd-b3e9-1afeff65be8c\") " pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:41.793162 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.793132 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-vz9sg"] Jun 09 03:12:41.795286 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:12:41.795250 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69d8535_cb27_4360_92e1_c481950bd98a.slice/crio-ade5e81cb0add5d85914f93dca901f1fabdc482e3c74df9281ff23c91d0dea8b WatchSource:0}: Error finding container ade5e81cb0add5d85914f93dca901f1fabdc482e3c74df9281ff23c91d0dea8b: Status 404 returned error can't find the container with id ade5e81cb0add5d85914f93dca901f1fabdc482e3c74df9281ff23c91d0dea8b Jun 09 03:12:41.832635 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.832600 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:41.971439 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:41.971408 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8c57d4bd7-9mmp9"] Jun 09 03:12:41.973966 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:12:41.973931 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac742c0a_ca0a_42dd_b3e9_1afeff65be8c.slice/crio-4dc9e428a659a13552e4ded016ece19c703b47e3cd0cfabcb1e354e97af8f646 WatchSource:0}: Error finding container 4dc9e428a659a13552e4ded016ece19c703b47e3cd0cfabcb1e354e97af8f646: Status 404 returned error can't find the container with id 4dc9e428a659a13552e4ded016ece19c703b47e3cd0cfabcb1e354e97af8f646 Jun 09 03:12:42.648953 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:42.648856 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" event={"ID":"ac742c0a-ca0a-42dd-b3e9-1afeff65be8c","Type":"ContainerStarted","Data":"4dc9e428a659a13552e4ded016ece19c703b47e3cd0cfabcb1e354e97af8f646"} Jun 09 03:12:42.651029 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:42.650941 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" event={"ID":"a69d8535-cb27-4360-92e1-c481950bd98a","Type":"ContainerStarted","Data":"ade5e81cb0add5d85914f93dca901f1fabdc482e3c74df9281ff23c91d0dea8b"} Jun 09 03:12:44.665956 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:44.665903 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" event={"ID":"a69d8535-cb27-4360-92e1-c481950bd98a","Type":"ContainerStarted","Data":"e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265"} Jun 09 03:12:44.666421 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:44.665984 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:44.668110 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:44.668050 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" event={"ID":"ac742c0a-ca0a-42dd-b3e9-1afeff65be8c","Type":"ContainerStarted","Data":"e103fc3f264830689f068feeee22d0fb2c3f26d233c392ea571446a62b72e51b"} Jun 09 03:12:44.668290 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:44.668210 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:44.690091 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:44.690025 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" podStartSLOduration=1.009648192 podStartE2EDuration="3.689998797s" totalImagesPullingTime="2.680350605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:12:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:12:44.687674586 +0000 UTC m=+798.597188933" watchObservedRunningTime="2026-06-09 03:12:44.689998797 +0000 UTC m=+798.599513148" Jun 09 03:12:44.709644 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:44.709588 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" podStartSLOduration=1.21926321 podStartE2EDuration="3.709571008s" totalImagesPullingTime="2.490307798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:12:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:12:44.707612093 +0000 UTC m=+798.617126441" watchObservedRunningTime="2026-06-09 03:12:44.709571008 +0000 UTC m=+798.619085354" Jun 09 03:12:55.679041 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:55.678952 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8c57d4bd7-9mmp9" Jun 09 03:12:55.679041 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:55.679032 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:55.732586 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:55.732551 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-vz9sg"] Jun 09 03:12:55.732786 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:55.732755 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" podUID="a69d8535-cb27-4360-92e1-c481950bd98a" containerName="manager" containerID="cri-o://e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265" gracePeriod=30 Jun 09 03:12:55.986393 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:55.986362 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:56.081645 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.081599 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j522d\" (UniqueName: \"kubernetes.io/projected/a69d8535-cb27-4360-92e1-c481950bd98a-kube-api-access-j522d\") pod \"a69d8535-cb27-4360-92e1-c481950bd98a\" (UID: \"a69d8535-cb27-4360-92e1-c481950bd98a\") " Jun 09 03:12:56.081645 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.081648 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a69d8535-cb27-4360-92e1-c481950bd98a-webhook-cert\") pod \"a69d8535-cb27-4360-92e1-c481950bd98a\" (UID: \"a69d8535-cb27-4360-92e1-c481950bd98a\") " Jun 09 03:12:56.084068 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.084028 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a69d8535-cb27-4360-92e1-c481950bd98a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a69d8535-cb27-4360-92e1-c481950bd98a" (UID: "a69d8535-cb27-4360-92e1-c481950bd98a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 03:12:56.084195 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.084085 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69d8535-cb27-4360-92e1-c481950bd98a-kube-api-access-j522d" (OuterVolumeSpecName: "kube-api-access-j522d") pod "a69d8535-cb27-4360-92e1-c481950bd98a" (UID: "a69d8535-cb27-4360-92e1-c481950bd98a"). InnerVolumeSpecName "kube-api-access-j522d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 03:12:56.182740 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.182692 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j522d\" (UniqueName: \"kubernetes.io/projected/a69d8535-cb27-4360-92e1-c481950bd98a-kube-api-access-j522d\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:12:56.182740 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.182732 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a69d8535-cb27-4360-92e1-c481950bd98a-webhook-cert\") on node \"ip-10-0-130-143.ec2.internal\" DevicePath \"\"" Jun 09 03:12:56.727329 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.727286 2564 generic.go:356] "Generic (PLEG): container finished" podID="a69d8535-cb27-4360-92e1-c481950bd98a" containerID="e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265" exitCode=0 Jun 09 03:12:56.727329 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.727330 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" Jun 09 03:12:56.727901 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.727396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" event={"ID":"a69d8535-cb27-4360-92e1-c481950bd98a","Type":"ContainerDied","Data":"e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265"} Jun 09 03:12:56.727901 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.727438 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdb77c66-vz9sg" event={"ID":"a69d8535-cb27-4360-92e1-c481950bd98a","Type":"ContainerDied","Data":"ade5e81cb0add5d85914f93dca901f1fabdc482e3c74df9281ff23c91d0dea8b"} Jun 09 03:12:56.727901 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.727478 2564 scope.go:117] "RemoveContainer" containerID="e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265" Jun 09 03:12:56.737802 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.737572 2564 scope.go:117] "RemoveContainer" containerID="e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265" Jun 09 03:12:56.737875 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:12:56.737817 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265\": container with ID starting with e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265 not found: ID does not exist" containerID="e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265" Jun 09 03:12:56.737875 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.737845 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265"} err="failed to get container status \"e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265\": rpc error: code = NotFound desc = could not find container \"e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265\": container with ID starting with e42d4076a843f6d317db2f81e724567e971d05e191376147c1ca8b8bc7c93265 not found: ID does not exist" Jun 09 03:12:56.741832 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.741802 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-vz9sg"] Jun 09 03:12:56.744693 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:56.744663 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-vz9sg"] Jun 09 03:12:58.551762 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:12:58.551726 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a69d8535-cb27-4360-92e1-c481950bd98a" path="/var/lib/kubelet/pods/a69d8535-cb27-4360-92e1-c481950bd98a/volumes" Jun 09 03:13:12.637217 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.637161 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-76cdcb8cdb-xqk2h"] Jun 09 03:13:12.637701 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.637569 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69d8535-cb27-4360-92e1-c481950bd98a" containerName="manager" Jun 09 03:13:12.637701 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.637581 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69d8535-cb27-4360-92e1-c481950bd98a" containerName="manager" Jun 09 03:13:12.637701 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.637659 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69d8535-cb27-4360-92e1-c481950bd98a" containerName="manager" Jun 09 03:13:12.640894 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.640873 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:12.642586 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.642562 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 09 03:13:12.651189 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.651160 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-76cdcb8cdb-xqk2h"] Jun 09 03:13:12.738236 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.738178 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-maas-api-tls\") pod \"maas-api-76cdcb8cdb-xqk2h\" (UID: \"9d69b6a8-c23a-4a81-9f57-e8e2147f713b\") " pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:12.738421 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.738320 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wslb\" (UniqueName: \"kubernetes.io/projected/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-kube-api-access-2wslb\") pod \"maas-api-76cdcb8cdb-xqk2h\" (UID: \"9d69b6a8-c23a-4a81-9f57-e8e2147f713b\") " pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:12.838976 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.838931 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wslb\" (UniqueName: \"kubernetes.io/projected/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-kube-api-access-2wslb\") pod \"maas-api-76cdcb8cdb-xqk2h\" (UID: \"9d69b6a8-c23a-4a81-9f57-e8e2147f713b\") " pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:12.839164 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.839032 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-maas-api-tls\") pod \"maas-api-76cdcb8cdb-xqk2h\" (UID: \"9d69b6a8-c23a-4a81-9f57-e8e2147f713b\") " pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:12.839164 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:13:12.839148 2564 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 09 03:13:12.839279 ip-10-0-130-143 kubenswrapper[2564]: E0609 03:13:12.839243 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-maas-api-tls podName:9d69b6a8-c23a-4a81-9f57-e8e2147f713b nodeName:}" failed. No retries permitted until 2026-06-09 03:13:13.339192663 +0000 UTC m=+827.248706993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-maas-api-tls") pod "maas-api-76cdcb8cdb-xqk2h" (UID: "9d69b6a8-c23a-4a81-9f57-e8e2147f713b") : secret "maas-api-serving-cert" not found Jun 09 03:13:12.848006 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:12.847976 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wslb\" (UniqueName: \"kubernetes.io/projected/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-kube-api-access-2wslb\") pod \"maas-api-76cdcb8cdb-xqk2h\" (UID: \"9d69b6a8-c23a-4a81-9f57-e8e2147f713b\") " pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:13.343984 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:13.343944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-maas-api-tls\") pod \"maas-api-76cdcb8cdb-xqk2h\" (UID: \"9d69b6a8-c23a-4a81-9f57-e8e2147f713b\") " pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:13.346596 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:13.346566 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d69b6a8-c23a-4a81-9f57-e8e2147f713b-maas-api-tls\") pod \"maas-api-76cdcb8cdb-xqk2h\" (UID: \"9d69b6a8-c23a-4a81-9f57-e8e2147f713b\") " pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:13.442830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:13.442791 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:13:13.553117 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:13.553061 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:13.706568 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:13.706537 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-76cdcb8cdb-xqk2h"] Jun 09 03:13:13.708923 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:13:13.708894 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d69b6a8_c23a_4a81_9f57_e8e2147f713b.slice/crio-9883ec75ca12b83bd9bf311bfb46fb9487128173bae313c4a6e5f26a94d46cce WatchSource:0}: Error finding container 9883ec75ca12b83bd9bf311bfb46fb9487128173bae313c4a6e5f26a94d46cce: Status 404 returned error can't find the container with id 9883ec75ca12b83bd9bf311bfb46fb9487128173bae313c4a6e5f26a94d46cce Jun 09 03:13:13.710148 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:13.710128 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 03:13:13.815383 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:13.815346 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" event={"ID":"9d69b6a8-c23a-4a81-9f57-e8e2147f713b","Type":"ContainerStarted","Data":"9883ec75ca12b83bd9bf311bfb46fb9487128173bae313c4a6e5f26a94d46cce"} Jun 09 03:13:15.827597 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:15.827559 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" event={"ID":"9d69b6a8-c23a-4a81-9f57-e8e2147f713b","Type":"ContainerStarted","Data":"c8dc66ef6b13e8fbf5f1e3f3835a9084f44ea0892a61d2c70b8f6e472f705473"} Jun 09 03:13:15.827987 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:15.827663 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:15.841612 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:15.841543 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" podStartSLOduration=2.262233868 podStartE2EDuration="3.841519245s" totalImagesPullingTime="1.579285377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:13:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:13:15.84103741 +0000 UTC m=+829.750551758" watchObservedRunningTime="2026-06-09 03:13:15.841519245 +0000 UTC m=+829.751033596" Jun 09 03:13:21.842919 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:21.842884 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-76cdcb8cdb-xqk2h" Jun 09 03:13:21.866706 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:21.866670 2564 generic.go:356] "Generic (PLEG): container finished" podID="cd782738-6218-4c10-ac8b-fbe789ae065e" containerID="6b7bee553bfbad89e978300e15cf77128fadaba7c3f29d94a38727e3c06fa493" exitCode=0 Jun 09 03:13:21.866903 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:21.866747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" event={"ID":"cd782738-6218-4c10-ac8b-fbe789ae065e","Type":"ContainerDied","Data":"6b7bee553bfbad89e978300e15cf77128fadaba7c3f29d94a38727e3c06fa493"} Jun 09 03:13:21.867286 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:21.867264 2564 scope.go:117] "RemoveContainer" containerID="6b7bee553bfbad89e978300e15cf77128fadaba7c3f29d94a38727e3c06fa493" Jun 09 03:13:21.868517 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:21.868487 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:13:21.868616 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:21.868583 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:13:21.868681 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:21.868656 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 09 03:13:22.873306 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:22.873267 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" event={"ID":"cd782738-6218-4c10-ac8b-fbe789ae065e","Type":"ContainerStarted","Data":"ef663f85b09a1b2762a827f0e0bcf137dfd2843ea6f6ea868bcee591350c1400"} Jun 09 03:13:23.684150 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:23.684112 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:13:23.685298 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:23.685271 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-kjtgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jun 09 03:13:23.685361 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:23.685328 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" podUID="cd782738-6218-4c10-ac8b-fbe789ae065e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jun 09 03:13:24.683646 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:24.683613 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-kjtgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jun 09 03:13:24.684030 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:24.683669 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" podUID="cd782738-6218-4c10-ac8b-fbe789ae065e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jun 09 03:13:25.683778 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:25.683743 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-kjtgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jun 09 03:13:25.684215 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:25.683802 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" podUID="cd782738-6218-4c10-ac8b-fbe789ae065e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jun 09 03:13:26.684010 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:26.683930 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-kjtgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jun 09 03:13:26.684010 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:26.683993 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" podUID="cd782738-6218-4c10-ac8b-fbe789ae065e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jun 09 03:13:27.688168 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:27.688137 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:13:27.688656 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:27.688437 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:13:27.689124 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:13:27.689106 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-kjtgw" Jun 09 03:14:27.491336 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:27.491296 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:14:33.208271 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:33.208232 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:14:44.575359 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.575300 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9"] Jun 09 03:14:44.578569 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.578546 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.580857 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.580832 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 09 03:14:44.580857 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.580848 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 09 03:14:44.581054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.580832 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tm2kp\"" Jun 09 03:14:44.581054 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.580832 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 09 03:14:44.590497 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.590474 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9"] Jun 09 03:14:44.687397 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.687368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.687571 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.687411 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.687571 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.687448 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.687571 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.687517 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.687571 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.687536 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.687721 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.687600 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9hd6\" (UniqueName: \"kubernetes.io/projected/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-kube-api-access-c9hd6\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788311 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788276 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788512 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788321 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788512 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788512 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788376 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788512 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788392 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788512 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788415 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9hd6\" (UniqueName: \"kubernetes.io/projected/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-kube-api-access-c9hd6\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788803 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788771 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788857 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788800 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.788857 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.788813 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.790724 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.790702 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.790961 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.790944 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.795006 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.794981 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9hd6\" (UniqueName: \"kubernetes.io/projected/4dd15e68-8914-4b2b-90b8-03cb717f7ae0-kube-api-access-c9hd6\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9\" (UID: \"4dd15e68-8914-4b2b-90b8-03cb717f7ae0\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:44.892018 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:44.891928 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:14:45.024331 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:45.024297 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9"] Jun 09 03:14:45.027715 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:14:45.027689 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd15e68_8914_4b2b_90b8_03cb717f7ae0.slice/crio-310dc26e0c4bd067ed7492146ce56251e5cc1141e2baa77a581bba971a66d783 WatchSource:0}: Error finding container 310dc26e0c4bd067ed7492146ce56251e5cc1141e2baa77a581bba971a66d783: Status 404 returned error can't find the container with id 310dc26e0c4bd067ed7492146ce56251e5cc1141e2baa77a581bba971a66d783 Jun 09 03:14:45.182837 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:45.182754 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:14:45.278880 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:45.278846 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" event={"ID":"4dd15e68-8914-4b2b-90b8-03cb717f7ae0","Type":"ContainerStarted","Data":"310dc26e0c4bd067ed7492146ce56251e5cc1141e2baa77a581bba971a66d783"} Jun 09 03:14:49.304635 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:49.304593 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" event={"ID":"4dd15e68-8914-4b2b-90b8-03cb717f7ae0","Type":"ContainerStarted","Data":"94bb755bd25be60d4ea4f30fb6d59ba2553d02e091dd30c3735813a9d6fa3bc1"} Jun 09 03:14:53.588978 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:53.588946 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:14:55.337681 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:55.337615 2564 generic.go:356] "Generic (PLEG): container finished" podID="4dd15e68-8914-4b2b-90b8-03cb717f7ae0" containerID="94bb755bd25be60d4ea4f30fb6d59ba2553d02e091dd30c3735813a9d6fa3bc1" exitCode=0 Jun 09 03:14:55.337681 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:55.337667 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" event={"ID":"4dd15e68-8914-4b2b-90b8-03cb717f7ae0","Type":"ContainerDied","Data":"94bb755bd25be60d4ea4f30fb6d59ba2553d02e091dd30c3735813a9d6fa3bc1"} Jun 09 03:14:57.356685 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:57.356652 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" event={"ID":"4dd15e68-8914-4b2b-90b8-03cb717f7ae0","Type":"ContainerStarted","Data":"14057e5119c90f11ebd87d269dc59845129855ff69cb2b2b7096986d9ea63aad"} Jun 09 03:14:57.373702 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:14:57.373656 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" podStartSLOduration=2.766708542 podStartE2EDuration="13.373640153s" totalImagesPullingTime="4.695900915s" totalInitContainerRuntime="5.911030696s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:14:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:14:57.371637585 +0000 UTC m=+931.281151933" watchObservedRunningTime="2026-06-09 03:14:57.373640153 +0000 UTC m=+931.283154500" Jun 09 03:15:04.892864 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:04.892776 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:15:04.892864 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:04.892847 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:15:04.905289 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:04.905266 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:15:05.404359 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:05.404328 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9" Jun 09 03:15:14.085339 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.085308 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq"] Jun 09 03:15:14.099462 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.099444 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.102684 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.102656 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 09 03:15:14.104272 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.104240 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq"] Jun 09 03:15:14.260156 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.260123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.260513 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.260490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf264\" (UniqueName: \"kubernetes.io/projected/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-kube-api-access-gf264\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.260706 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.260663 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.260859 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.260836 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.260980 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.260891 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.261050 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.261009 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.361776 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.361691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.361776 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.361738 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.361955 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.361801 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf264\" (UniqueName: \"kubernetes.io/projected/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-kube-api-access-gf264\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.361955 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.361842 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.362046 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.361952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.362369 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.362316 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.362508 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.362471 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.362572 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.362547 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.367223 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.362659 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.367223 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.364972 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.367223 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.365545 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.371576 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.371554 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf264\" (UniqueName: \"kubernetes.io/projected/87e255dd-44b2-4e8e-862b-edb2a5ed7f8a-kube-api-access-gf264\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq\" (UID: \"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.411510 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.411476 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:14.544424 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.544398 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq"] Jun 09 03:15:14.546732 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:15:14.546705 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e255dd_44b2_4e8e_862b_edb2a5ed7f8a.slice/crio-46d62399e84bed9e18cdd18342e676bc5272c8b836a6bdbe18fde93a35c03b93 WatchSource:0}: Error finding container 46d62399e84bed9e18cdd18342e676bc5272c8b836a6bdbe18fde93a35c03b93: Status 404 returned error can't find the container with id 46d62399e84bed9e18cdd18342e676bc5272c8b836a6bdbe18fde93a35c03b93 Jun 09 03:15:14.886521 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:14.886488 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:15:15.441770 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:15.441727 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" event={"ID":"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a","Type":"ContainerStarted","Data":"bc422b812b63c2d8e1a10ba9deaab0ecb689a88e76046c320d4c9f51a0039e8e"} Jun 09 03:15:15.442222 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:15.441784 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" event={"ID":"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a","Type":"ContainerStarted","Data":"46d62399e84bed9e18cdd18342e676bc5272c8b836a6bdbe18fde93a35c03b93"} Jun 09 03:15:16.449407 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:16.449375 2564 generic.go:356] "Generic (PLEG): container finished" podID="87e255dd-44b2-4e8e-862b-edb2a5ed7f8a" containerID="bc422b812b63c2d8e1a10ba9deaab0ecb689a88e76046c320d4c9f51a0039e8e" exitCode=0 Jun 09 03:15:16.449801 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:16.449461 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" event={"ID":"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a","Type":"ContainerDied","Data":"bc422b812b63c2d8e1a10ba9deaab0ecb689a88e76046c320d4c9f51a0039e8e"} Jun 09 03:15:17.455644 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:17.455604 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" event={"ID":"87e255dd-44b2-4e8e-862b-edb2a5ed7f8a","Type":"ContainerStarted","Data":"2750d4bd0c7bd3dfd1dc4eaf2e1a9ddcacbd82e1dda3783eb21fd6d3954db9bf"} Jun 09 03:15:17.478825 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:17.472673 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" podStartSLOduration=1.891588428 podStartE2EDuration="3.472658677s" totalImagesPullingTime="216.158472ms" totalInitContainerRuntime="1.364911777s" isStatefulPod=false podCreationTimestamp="2026-06-09 03:15:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:15:17.47061 +0000 UTC m=+951.380124346" watchObservedRunningTime="2026-06-09 03:15:17.472658677 +0000 UTC m=+951.382173083" Jun 09 03:15:19.284798 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.284757 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf"] Jun 09 03:15:19.288489 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.288467 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.290621 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.290600 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 09 03:15:19.297041 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.297006 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf"] Jun 09 03:15:19.413762 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.413728 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.413762 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.413764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfrlx\" (UniqueName: \"kubernetes.io/projected/e6445676-6deb-4bb1-934b-d7b747dc5008-kube-api-access-mfrlx\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.414027 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.413799 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.414027 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.413917 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6445676-6deb-4bb1-934b-d7b747dc5008-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.414027 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.413965 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.414027 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.414025 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.514977 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.514948 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6445676-6deb-4bb1-934b-d7b747dc5008-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.515174 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.514993 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.515174 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.515034 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.515174 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.515105 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.515174 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.515130 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfrlx\" (UniqueName: \"kubernetes.io/projected/e6445676-6deb-4bb1-934b-d7b747dc5008-kube-api-access-mfrlx\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.515174 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.515173 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.515501 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.515462 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.515588 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.515570 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.515648 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.515622 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.517327 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.517307 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6445676-6deb-4bb1-934b-d7b747dc5008-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.517498 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.517481 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6445676-6deb-4bb1-934b-d7b747dc5008-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.522161 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.522137 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfrlx\" (UniqueName: \"kubernetes.io/projected/e6445676-6deb-4bb1-934b-d7b747dc5008-kube-api-access-mfrlx\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-vqwjf\" (UID: \"e6445676-6deb-4bb1-934b-d7b747dc5008\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.602365 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.602289 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" Jun 09 03:15:19.736648 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:19.736621 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf"] Jun 09 03:15:19.738377 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:15:19.738349 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6445676_6deb_4bb1_934b_d7b747dc5008.slice/crio-3fef47ad29fc8f2dfb8424d2d2b044c5c2778644d409c4a3d48c80ba1967bca7 WatchSource:0}: Error finding container 3fef47ad29fc8f2dfb8424d2d2b044c5c2778644d409c4a3d48c80ba1967bca7: Status 404 returned error can't find the container with id 3fef47ad29fc8f2dfb8424d2d2b044c5c2778644d409c4a3d48c80ba1967bca7 Jun 09 03:15:20.292360 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:20.292328 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4gjhs"] Jun 09 03:15:20.471902 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:20.471869 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" event={"ID":"e6445676-6deb-4bb1-934b-d7b747dc5008","Type":"ContainerStarted","Data":"67bfb3eb21e5085656d1f0252e480bc581abefca173f6a2fcec546db1b19a060"} Jun 09 03:15:20.472060 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:20.471911 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-vqwjf" event={"ID":"e6445676-6deb-4bb1-934b-d7b747dc5008","Type":"ContainerStarted","Data":"3fef47ad29fc8f2dfb8424d2d2b044c5c2778644d409c4a3d48c80ba1967bca7"} Jun 09 03:15:24.412315 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:24.412273 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:24.412728 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:24.412350 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:24.425216 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:24.425178 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:15:24.504061 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:15:24.504028 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq" Jun 09 03:21:25.730579 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:25.730537 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-vqwjf_e6445676-6deb-4bb1-934b-d7b747dc5008/storage-initializer/0.log" Jun 09 03:21:31.163915 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:31.163887 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6d869d9c69-mpkfv_42295f07-c81a-444a-858e-f6a175e65a4a/nginx/0.log" Jun 09 03:21:31.621601 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:31.621556 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-76cdcb8cdb-xqk2h_9d69b6a8-c23a-4a81-9f57-e8e2147f713b/maas-api/0.log" Jun 09 03:21:31.851570 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:31.851544 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8c57d4bd7-9mmp9_ac742c0a-ca0a-42dd-b3e9-1afeff65be8c/manager/0.log" Jun 09 03:21:32.431578 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:32.431552 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bfbb9599-x9x65_10c1ac23-2fe7-44ab-9708-f46bed3d5662/copy-manifests/0.log" Jun 09 03:21:32.444234 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:32.444194 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bfbb9599-x9x65_10c1ac23-2fe7-44ab-9708-f46bed3d5662/manager/0.log" Jun 09 03:21:34.029924 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:34.029886 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-j5d52_270883d6-8776-4d80-b736-b71c9d96a2b7/manager/0.log" Jun 09 03:21:34.252284 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:34.252254 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-9s4bt_b1970ad6-7faf-4ec2-8351-6660e1a96495/registry-server/0.log" Jun 09 03:21:34.370499 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:34.370429 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-j5bgl_7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb/manager/0.log" Jun 09 03:21:34.488682 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:34.488654 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4gjhs_da0d94b3-702e-4fb9-8de6-245b4a23909a/limitador/0.log" Jun 09 03:21:34.603021 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:34.602979 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8sbzg_c42662c6-e708-4935-b775-3f43ed4497bb/manager/0.log" Jun 09 03:21:34.955735 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:34.955712 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979544fr6_89f322a8-4802-4cb4-b690-4a89dc3deed0/istio-proxy/0.log" Jun 09 03:21:35.424960 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:35.424880 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-kjtgw_cd782738-6218-4c10-ac8b-fbe789ae065e/istio-proxy/1.log" Jun 09 03:21:35.752654 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:35.752630 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74b85654cf-49rnq_d030d15e-f898-4646-ba0b-f21734c72a89/router/0.log" Jun 09 03:21:36.322074 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:36.322041 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9_4dd15e68-8914-4b2b-90b8-03cb717f7ae0/main/0.log" Jun 09 03:21:36.328057 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:36.328041 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-hc6f9_4dd15e68-8914-4b2b-90b8-03cb717f7ae0/storage-initializer/0.log" Jun 09 03:21:36.553785 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:36.553763 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq_87e255dd-44b2-4e8e-862b-edb2a5ed7f8a/storage-initializer/0.log" Jun 09 03:21:36.560659 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:36.560622 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-7mwwq_87e255dd-44b2-4e8e-862b-edb2a5ed7f8a/main/0.log" Jun 09 03:21:36.672568 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:36.672461 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-vqwjf_e6445676-6deb-4bb1-934b-d7b747dc5008/storage-initializer/0.log" Jun 09 03:21:43.875328 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:43.875299 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mjbdx_125f4810-5dbf-4e8f-9eb0-ecd8e22564d4/global-pull-secret-syncer/0.log" Jun 09 03:21:43.969830 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:43.969803 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xhqhk_a2adb943-7cd5-43df-bdce-3309518d60c5/konnectivity-agent/0.log" Jun 09 03:21:43.994295 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:43.994258 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-143.ec2.internal_ff922de98c02d8203e86d43a866df755/haproxy/0.log" Jun 09 03:21:48.844601 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:48.844497 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-j5d52_270883d6-8776-4d80-b736-b71c9d96a2b7/manager/0.log" Jun 09 03:21:48.918467 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:48.918420 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-9s4bt_b1970ad6-7faf-4ec2-8351-6660e1a96495/registry-server/0.log" Jun 09 03:21:48.946590 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:48.946552 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-j5bgl_7867e3d0-6e5a-4d19-abbe-7dd6b30a3eeb/manager/0.log" Jun 09 03:21:48.965664 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:48.965643 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4gjhs_da0d94b3-702e-4fb9-8de6-245b4a23909a/limitador/0.log" Jun 09 03:21:48.998290 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:48.998272 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8sbzg_c42662c6-e708-4935-b775-3f43ed4497bb/manager/0.log" Jun 09 03:21:51.040525 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.040500 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-c4b5k_e0a7bf72-0ca0-491a-8445-6757ea247410/kube-state-metrics/0.log" Jun 09 03:21:51.057942 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.057917 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-c4b5k_e0a7bf72-0ca0-491a-8445-6757ea247410/kube-rbac-proxy-main/0.log" Jun 09 03:21:51.081595 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.081571 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-c4b5k_e0a7bf72-0ca0-491a-8445-6757ea247410/kube-rbac-proxy-self/0.log" Jun 09 03:21:51.175618 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.175596 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k2ht_0a6cfce7-ffb6-4bc3-ba83-d35d2482678c/node-exporter/0.log" Jun 09 03:21:51.195614 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.195595 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k2ht_0a6cfce7-ffb6-4bc3-ba83-d35d2482678c/kube-rbac-proxy/0.log" Jun 09 03:21:51.216412 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.216395 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6k2ht_0a6cfce7-ffb6-4bc3-ba83-d35d2482678c/init-textfile/0.log" Jun 09 03:21:51.504666 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.504642 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53d507e9-7e04-4da0-b18b-6b75a8743ee5/prometheus/0.log" Jun 09 03:21:51.528373 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.528349 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53d507e9-7e04-4da0-b18b-6b75a8743ee5/config-reloader/0.log" Jun 09 03:21:51.549669 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.549610 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53d507e9-7e04-4da0-b18b-6b75a8743ee5/thanos-sidecar/0.log" Jun 09 03:21:51.570617 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.570597 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53d507e9-7e04-4da0-b18b-6b75a8743ee5/kube-rbac-proxy-web/0.log" Jun 09 03:21:51.594887 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.594868 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53d507e9-7e04-4da0-b18b-6b75a8743ee5/kube-rbac-proxy/0.log" Jun 09 03:21:51.626114 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.626097 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53d507e9-7e04-4da0-b18b-6b75a8743ee5/kube-rbac-proxy-thanos/0.log" Jun 09 03:21:51.648472 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.648455 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53d507e9-7e04-4da0-b18b-6b75a8743ee5/init-config-reloader/0.log" Jun 09 03:21:51.720372 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.720340 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-9bwz9_10d21e65-2a1a-4e99-bf92-f3385a1ab81b/prometheus-operator-admission-webhook/0.log" Jun 09 03:21:51.836309 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.836239 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/thanos-query/0.log" Jun 09 03:21:51.860397 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.860379 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/kube-rbac-proxy-web/0.log" Jun 09 03:21:51.886112 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.886093 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/kube-rbac-proxy/0.log" Jun 09 03:21:51.907352 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.907328 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/prom-label-proxy/0.log" Jun 09 03:21:51.930619 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.930602 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/kube-rbac-proxy-rules/0.log" Jun 09 03:21:51.953222 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:51.953182 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dc7d7fff-wlw9r_65b0ffdc-a288-4086-a2a3-e5c0227ee50b/kube-rbac-proxy-metrics/0.log" Jun 09 03:21:53.708773 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.708743 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr"] Jun 09 03:21:53.712625 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.712607 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.714297 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.714272 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vbctj\"/\"default-dockercfg-4jj8c\"" Jun 09 03:21:53.714423 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.714319 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbctj\"/\"kube-root-ca.crt\"" Jun 09 03:21:53.714803 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.714774 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbctj\"/\"openshift-service-ca.crt\"" Jun 09 03:21:53.720364 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.719869 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr"] Jun 09 03:21:53.851249 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.851215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcv2p\" (UniqueName: \"kubernetes.io/projected/247c6505-dd09-4a85-9732-ffe6a17e9238-kube-api-access-vcv2p\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.851407 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.851284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-lib-modules\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.851407 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.851313 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-proc\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.851481 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.851404 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-sys\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.851481 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.851453 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-podres\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952125 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952092 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-sys\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952327 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952140 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-podres\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952327 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952170 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcv2p\" (UniqueName: \"kubernetes.io/projected/247c6505-dd09-4a85-9732-ffe6a17e9238-kube-api-access-vcv2p\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952327 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-lib-modules\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952327 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952225 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-sys\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952327 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952265 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-proc\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952606 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952325 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-podres\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952606 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952332 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-lib-modules\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.952606 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.952366 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/247c6505-dd09-4a85-9732-ffe6a17e9238-proc\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:53.958534 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:53.958508 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcv2p\" (UniqueName: \"kubernetes.io/projected/247c6505-dd09-4a85-9732-ffe6a17e9238-kube-api-access-vcv2p\") pod \"perf-node-gather-daemonset-95jmr\" (UID: \"247c6505-dd09-4a85-9732-ffe6a17e9238\") " pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:54.023435 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:54.023359 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:54.149908 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:54.149879 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr"] Jun 09 03:21:54.151617 ip-10-0-130-143 kubenswrapper[2564]: W0609 03:21:54.151588 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod247c6505_dd09_4a85_9732_ffe6a17e9238.slice/crio-1da31ccb9e5cdb4de650492f9c347db2a9bf9852aa835574502123624478a532 WatchSource:0}: Error finding container 1da31ccb9e5cdb4de650492f9c347db2a9bf9852aa835574502123624478a532: Status 404 returned error can't find the container with id 1da31ccb9e5cdb4de650492f9c347db2a9bf9852aa835574502123624478a532 Jun 09 03:21:54.352065 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:54.351986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" event={"ID":"247c6505-dd09-4a85-9732-ffe6a17e9238","Type":"ContainerStarted","Data":"c121e66cc31b3ec9314460bb8015408392a91ff896edc0b97d16d0c9bcd2e172"} Jun 09 03:21:54.352065 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:54.352034 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" event={"ID":"247c6505-dd09-4a85-9732-ffe6a17e9238","Type":"ContainerStarted","Data":"1da31ccb9e5cdb4de650492f9c347db2a9bf9852aa835574502123624478a532"} Jun 09 03:21:54.352253 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:54.352075 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:21:54.365250 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:54.365173 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" podStartSLOduration=1.365159413 podStartE2EDuration="1.365159413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 03:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 03:21:54.364350655 +0000 UTC m=+1348.273865014" watchObservedRunningTime="2026-06-09 03:21:54.365159413 +0000 UTC m=+1348.274673754" Jun 09 03:21:56.158769 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:56.158742 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8bgc7_acd95e7e-686a-4ba0-990f-38a36a91ba27/dns/0.log" Jun 09 03:21:56.183300 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:56.183279 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8bgc7_acd95e7e-686a-4ba0-990f-38a36a91ba27/kube-rbac-proxy/0.log" Jun 09 03:21:56.298808 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:56.298777 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dg64b_b73b4444-c489-4e85-aabd-b69d0c1553b3/dns-node-resolver/0.log" Jun 09 03:21:56.912008 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:56.911982 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vww9x_e2fa6373-a391-4405-8a41-fb03ae52a042/node-ca/0.log" Jun 09 03:21:57.821352 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:57.821304 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979544fr6_89f322a8-4802-4cb4-b690-4a89dc3deed0/istio-proxy/0.log" Jun 09 03:21:57.934173 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:57.934148 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-kjtgw_cd782738-6218-4c10-ac8b-fbe789ae065e/istio-proxy/1.log" Jun 09 03:21:57.934564 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:57.934547 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-kjtgw_cd782738-6218-4c10-ac8b-fbe789ae065e/istio-proxy/0.log" Jun 09 03:21:57.993610 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:57.993577 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74b85654cf-49rnq_d030d15e-f898-4646-ba0b-f21734c72a89/router/0.log" Jun 09 03:21:58.610462 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:58.610432 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sc8p8_92273549-70ad-4aab-a95e-7b361eed8781/serve-healthcheck-canary/0.log" Jun 09 03:21:59.259789 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:59.259759 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xcnf2_810d93c5-2966-46af-8522-eb4921bf0dbe/kube-rbac-proxy/0.log" Jun 09 03:21:59.279609 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:59.279589 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xcnf2_810d93c5-2966-46af-8522-eb4921bf0dbe/exporter/0.log" Jun 09 03:21:59.299750 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:21:59.299716 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xcnf2_810d93c5-2966-46af-8522-eb4921bf0dbe/extractor/0.log" Jun 09 03:22:00.366325 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:00.366300 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vbctj/perf-node-gather-daemonset-95jmr" Jun 09 03:22:01.452450 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:01.452421 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6d869d9c69-mpkfv_42295f07-c81a-444a-858e-f6a175e65a4a/nginx/0.log" Jun 09 03:22:01.549618 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:01.549597 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-76cdcb8cdb-xqk2h_9d69b6a8-c23a-4a81-9f57-e8e2147f713b/maas-api/0.log" Jun 09 03:22:01.606937 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:01.606910 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8c57d4bd7-9mmp9_ac742c0a-ca0a-42dd-b3e9-1afeff65be8c/manager/0.log" Jun 09 03:22:01.802327 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:01.802250 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bfbb9599-x9x65_10c1ac23-2fe7-44ab-9708-f46bed3d5662/manager/0.log" Jun 09 03:22:01.819608 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:01.819584 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bfbb9599-x9x65_10c1ac23-2fe7-44ab-9708-f46bed3d5662/copy-manifests/0.log" Jun 09 03:22:03.542024 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:03.541955 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59599dfcbf-n6r28_2d2803b7-6583-4a0c-bb4a-666d4377b47b/manager/0.log" Jun 09 03:22:08.619997 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:08.619967 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979544fr6_89f322a8-4802-4cb4-b690-4a89dc3deed0/istio-proxy/0.log" Jun 09 03:22:08.704746 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:08.704716 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-kjtgw_cd782738-6218-4c10-ac8b-fbe789ae065e/istio-proxy/1.log" Jun 09 03:22:08.705137 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:08.705123 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-kjtgw_cd782738-6218-4c10-ac8b-fbe789ae065e/istio-proxy/0.log" Jun 09 03:22:08.742724 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:08.742691 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74b85654cf-49rnq_d030d15e-f898-4646-ba0b-f21734c72a89/router/0.log" Jun 09 03:22:09.810084 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:09.810006 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979544fr6_89f322a8-4802-4cb4-b690-4a89dc3deed0/istio-proxy/0.log" Jun 09 03:22:09.897631 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:09.897608 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-kjtgw_cd782738-6218-4c10-ac8b-fbe789ae065e/istio-proxy/1.log" Jun 09 03:22:09.897937 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:09.897916 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-kjtgw_cd782738-6218-4c10-ac8b-fbe789ae065e/istio-proxy/0.log" Jun 09 03:22:09.943640 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:09.943615 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74b85654cf-49rnq_d030d15e-f898-4646-ba0b-f21734c72a89/router/0.log" Jun 09 03:22:10.374948 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.374926 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-789lw_348a74fc-1fea-4472-ae4a-d26df51b1993/kube-multus-additional-cni-plugins/0.log" Jun 09 03:22:10.396425 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.396402 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-789lw_348a74fc-1fea-4472-ae4a-d26df51b1993/egress-router-binary-copy/0.log" Jun 09 03:22:10.418314 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.418289 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-789lw_348a74fc-1fea-4472-ae4a-d26df51b1993/cni-plugins/0.log" Jun 09 03:22:10.439084 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.439068 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-789lw_348a74fc-1fea-4472-ae4a-d26df51b1993/bond-cni-plugin/0.log" Jun 09 03:22:10.459698 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.459679 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-789lw_348a74fc-1fea-4472-ae4a-d26df51b1993/routeoverride-cni/0.log" Jun 09 03:22:10.480052 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.480031 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-789lw_348a74fc-1fea-4472-ae4a-d26df51b1993/whereabouts-cni-bincopy/0.log" Jun 09 03:22:10.500013 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.499992 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-789lw_348a74fc-1fea-4472-ae4a-d26df51b1993/whereabouts-cni/0.log" Jun 09 03:22:10.665194 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.665133 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-r2p87_1dc94e9a-d865-4694-9a6a-41f628b714de/kube-multus/0.log" Jun 09 03:22:10.724460 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.724435 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l65ml_9df3f892-882c-409c-a7ac-44e81fe9c003/network-metrics-daemon/0.log" Jun 09 03:22:10.743750 ip-10-0-130-143 kubenswrapper[2564]: I0609 03:22:10.743727 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l65ml_9df3f892-882c-409c-a7ac-44e81fe9c003/kube-rbac-proxy/0.log"