Jun 11 13:08:03.367822 ip-10-0-141-252 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 11 13:08:03.367856 ip-10-0-141-252 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 11 13:08:03.367864 ip-10-0-141-252 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 11 13:08:03.368173 ip-10-0-141-252 systemd[1]: Failed to start Kubernetes Kubelet. Jun 11 13:08:13.031218 ip-10-0-141-252 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jun 11 13:08:13.031228 ip-10-0-141-252 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5c05de1cedc344ef8ad3c2e8c035d5be -- Jun 11 13:10:36.631925 ip-10-0-141-252 systemd[1]: Starting Kubernetes Kubelet... Jun 11 13:10:37.051410 ip-10-0-141-252 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 13:10:37.051410 ip-10-0-141-252 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 13:10:37.051410 ip-10-0-141-252 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 13:10:37.051410 ip-10-0-141-252 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 13:10:37.051410 ip-10-0-141-252 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 13:10:37.053918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.053830 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 13:10:37.059784 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059769 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 13:10:37.059784 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059784 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059787 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059791 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059794 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059798 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059800 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059803 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059806 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059809 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059811 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059819 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059822 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059824 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059827 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059830 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059833 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059835 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059838 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059840 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059844 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 13:10:37.059847 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059846 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059849 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059851 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059855 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059858 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059861 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059864 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059866 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059868 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059871 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059873 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059876 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059879 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059882 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059885 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059887 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059890 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059892 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059894 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059897 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 13:10:37.060327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059899 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059902 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059905 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059907 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059910 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059912 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059914 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059918 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059920 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059922 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059925 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059927 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059930 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059932 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059935 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059946 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059949 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059953 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059957 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 13:10:37.060815 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059961 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059964 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059967 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059970 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059973 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059976 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059978 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059981 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059983 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059985 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059988 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059990 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059992 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059995 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.059997 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060001 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060003 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060006 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060008 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060011 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 13:10:37.061273 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060014 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060016 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060018 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060021 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060023 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060026 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060031 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060035 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060038 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060041 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060043 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060046 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060048 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060051 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060054 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060056 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060059 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060061 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060063 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060783 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 13:10:37.061763 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060790 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060793 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060795 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060798 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060801 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060803 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060806 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060809 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060812 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060815 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060817 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060820 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060822 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060825 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060828 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060830 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060834 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060838 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060841 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060846 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 13:10:37.062277 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060848 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060851 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060854 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060857 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060859 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060862 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060865 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060868 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060870 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060872 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060875 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060877 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060880 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060882 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060885 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060887 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060890 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060892 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060895 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060898 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 13:10:37.062785 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060900 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060903 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060905 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060908 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060911 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060913 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060915 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060918 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060920 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060923 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060925 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060928 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060931 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060934 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060936 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060939 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060941 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060944 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060946 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060949 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 13:10:37.063327 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060951 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060954 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060956 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060959 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060963 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060966 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060969 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060972 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060975 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060978 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060980 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060983 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060985 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060987 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060990 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060993 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060995 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.060998 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061000 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 13:10:37.063861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061003 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061005 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061008 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061010 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061013 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061015 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061018 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061020 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061022 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061025 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061027 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061029 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061032 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061035 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061037 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061040 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061042 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061045 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061048 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061418 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 13:10:37.064357 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061424 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061427 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061430 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061433 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061436 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061439 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061442 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061445 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061448 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061451 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061454 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061456 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061459 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061462 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061465 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061467 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061470 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061473 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061475 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061477 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 13:10:37.064861 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061480 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061483 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061485 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061488 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061491 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061493 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061496 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061498 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061500 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061503 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061505 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061507 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061510 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061512 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061515 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061517 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061520 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061522 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061524 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061527 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 13:10:37.065342 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061530 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061532 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061535 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061537 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061540 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061543 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061545 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061548 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061550 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061552 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061555 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061557 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061560 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061562 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061564 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061567 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061570 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061573 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061577 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 13:10:37.065857 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061579 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061582 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061585 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061587 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061590 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061592 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061595 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061597 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061599 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061602 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061604 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061607 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061609 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061611 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061633 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061636 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061638 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061643 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061647 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061650 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 13:10:37.066323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061656 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061659 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061661 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061664 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061666 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061669 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061672 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061675 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061677 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061680 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061682 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061685 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061687 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061690 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061692 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061696 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061699 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061702 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.061704 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061795 2563 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061805 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 13:10:37.066830 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061813 2563 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061819 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061825 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061831 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061841 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061845 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061849 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061852 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061856 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061859 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061862 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061866 2563 flags.go:64] FLAG: --cgroup-root="" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061869 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061872 2563 flags.go:64] FLAG: --client-ca-file="" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061875 2563 flags.go:64] FLAG: --cloud-provider="external" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061878 2563 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061882 2563 flags.go:64] FLAG: --cluster-domain="" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061885 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061889 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061892 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061895 2563 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061899 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061902 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061905 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 13:10:37.067327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061908 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061911 2563 flags.go:64] FLAG: --contention-profiling="false" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061914 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061917 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061920 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061923 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061927 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061930 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061933 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061936 2563 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061939 2563 flags.go:64] FLAG: --enable-server="true" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061942 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061947 2563 flags.go:64] FLAG: --event-burst="100" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061950 2563 flags.go:64] FLAG: --event-qps="50" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061952 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061956 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061959 2563 flags.go:64] FLAG: --eviction-hard="" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061963 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061966 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061973 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061976 2563 flags.go:64] FLAG: --eviction-soft="" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061979 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061982 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061985 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061988 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 13:10:37.067918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061990 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061993 2563 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.061996 2563 flags.go:64] FLAG: --feature-gates="" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062000 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062003 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062006 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062009 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062012 2563 flags.go:64] FLAG: --healthz-port="10248" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062015 2563 flags.go:64] FLAG: --help="false" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062017 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062020 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062023 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062026 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062029 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062032 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062035 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062038 2563 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062041 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062043 2563 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062047 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062050 2563 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062053 2563 flags.go:64] FLAG: --kube-reserved="" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062056 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062059 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 13:10:37.068588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062062 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062064 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062068 2563 flags.go:64] FLAG: --lock-file="" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062071 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062074 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062077 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062082 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062085 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062087 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062090 2563 flags.go:64] FLAG: --logging-format="text" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062093 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062096 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062099 2563 flags.go:64] FLAG: --manifest-url="" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062102 2563 flags.go:64] FLAG: --manifest-url-header="" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062106 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062109 2563 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062113 2563 flags.go:64] FLAG: --max-pods="110" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062116 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062119 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062122 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062125 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062128 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062131 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062133 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062146 2563 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 13:10:37.069202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062149 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062152 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062156 2563 flags.go:64] FLAG: --pod-cidr="" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062159 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062164 2563 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062167 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062170 2563 flags.go:64] FLAG: --pods-per-core="0" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062173 2563 flags.go:64] FLAG: --port="10250" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062176 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062180 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00b9b249032b9737e" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062183 2563 flags.go:64] FLAG: --qos-reserved="" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062186 2563 flags.go:64] FLAG: --read-only-port="10255" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062189 2563 flags.go:64] FLAG: --register-node="true" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062192 2563 flags.go:64] FLAG: --register-with-taints="" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062195 2563 flags.go:64] FLAG: --registry-burst="10" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062198 2563 flags.go:64] FLAG: --registry-qps="5" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062200 2563 flags.go:64] FLAG: --reserved-cpus="" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062203 2563 flags.go:64] FLAG: --reserved-memory="" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062207 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062210 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062213 2563 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062215 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062218 2563 flags.go:64] FLAG: --runonce="false" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062221 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062224 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 13:10:37.069865 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062227 2563 flags.go:64] FLAG: --seccomp-default="false" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062230 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062232 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062235 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062238 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062241 2563 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062244 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062247 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062249 2563 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062252 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062257 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062260 2563 flags.go:64] FLAG: --system-cgroups="" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062263 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062268 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062271 2563 flags.go:64] FLAG: --tls-cert-file="" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062274 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062279 2563 flags.go:64] FLAG: --tls-min-version="" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062283 2563 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062286 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062289 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062292 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062295 2563 flags.go:64] FLAG: --v="2" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062298 2563 flags.go:64] FLAG: --version="false" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062302 2563 flags.go:64] FLAG: --vmodule="" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062306 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 13:10:37.070463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.062309 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062428 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062432 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062435 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062438 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062441 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062444 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062446 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062449 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062452 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062454 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062457 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062459 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062462 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062464 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062466 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062469 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062473 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062475 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062478 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 13:10:37.071079 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062480 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062483 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062487 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062491 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062495 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062498 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062501 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062504 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062507 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062509 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062513 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062517 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062520 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062523 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062526 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062528 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062530 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062533 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062536 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 13:10:37.071593 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062538 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062541 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062543 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062546 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062548 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062550 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062553 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062555 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062558 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062560 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062564 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062566 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062569 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062571 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062574 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062576 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062580 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062598 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062603 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062607 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 13:10:37.072164 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062611 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062614 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062628 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062631 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062634 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062637 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062639 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062642 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062645 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062649 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062652 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062655 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062659 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062661 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062664 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062666 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062669 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062672 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062675 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 13:10:37.072710 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062677 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062680 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062682 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062687 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062690 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062692 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062695 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062698 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062701 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062705 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062707 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062711 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062714 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062717 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062719 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062722 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062725 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062727 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062730 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062732 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 13:10:37.073185 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062735 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.062737 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.063416 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.070037 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.070054 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.070073 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.070079 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070137 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070142 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070145 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070148 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070151 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070154 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070157 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070160 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 13:10:37.073671 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070163 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070165 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070168 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070170 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070173 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070176 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070178 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070181 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070183 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070186 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070189 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070191 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070194 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070197 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070199 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070202 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070204 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070207 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070209 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070212 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 13:10:37.074052 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070215 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070218 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070221 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070224 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070226 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070230 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070235 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070237 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070240 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070243 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070246 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070249 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070252 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070254 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070257 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070260 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070262 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070266 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070270 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 13:10:37.074536 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070273 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070276 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070278 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070281 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070284 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070288 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070290 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070293 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070295 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070298 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070301 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070303 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070306 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070309 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070311 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070314 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070317 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070319 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070322 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070324 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 13:10:37.075003 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070326 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070329 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070331 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070334 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070336 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070339 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070341 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070344 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070346 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070348 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070351 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070353 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070356 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070359 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070361 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070364 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070366 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070369 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070371 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070374 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 13:10:37.075478 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070376 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070379 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070381 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070384 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070387 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070389 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070392 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070394 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070399 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070402 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070405 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070408 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.070413 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070524 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 13:10:37.075981 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070529 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070532 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070535 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070538 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070542 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070546 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070548 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070551 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070553 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070556 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070558 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070561 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070563 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070567 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070569 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070572 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070574 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070577 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070581 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 13:10:37.076323 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070584 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070588 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070591 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070594 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070598 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070601 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070603 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070606 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070609 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070612 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070614 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070632 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070635 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070638 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070641 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070644 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070646 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070649 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070651 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 13:10:37.076831 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070654 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070656 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070659 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070661 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070664 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070666 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070669 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070671 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070674 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070676 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070679 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070681 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070683 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070686 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070688 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070690 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070693 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070696 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070699 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070701 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 13:10:37.077294 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070704 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070706 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070710 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070712 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070715 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070718 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070720 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070723 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070725 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070728 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070730 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070733 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070735 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070755 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070759 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070762 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070765 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070767 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070770 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070773 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 13:10:37.077807 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070775 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070778 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070780 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070782 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070785 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070787 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070790 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070792 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070795 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070797 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070800 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070802 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070805 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070807 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070810 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070813 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070816 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070819 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070821 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 13:10:37.078272 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:37.070824 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 13:10:37.078735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.070829 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 13:10:37.078735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.070939 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 13:10:37.078735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.072929 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 13:10:37.078735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.074588 2563 server.go:1019] "Starting client certificate rotation" Jun 11 13:10:37.078735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.074656 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 13:10:37.078735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.074695 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 13:10:37.078735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.077525 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 13:10:37.085787 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.085763 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 13:10:37.096188 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.096170 2563 log.go:25] "Validated CRI v1 runtime API" Jun 11 13:10:37.098868 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.098851 2563 log.go:25] "Validated CRI v1 image API" Jun 11 13:10:37.099663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.099647 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 13:10:37.104101 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.104082 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 13:10:37.105942 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.105917 2563 fs.go:135] Filesystem UUIDs: map[5ac95e04-7a75-4662-a2be-73ca91868888:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 db7c423f-3289-4811-ac67-afe639d89426:/dev/nvme0n1p4] Jun 11 13:10:37.105942 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.105935 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 13:10:37.110996 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.110892 2563 manager.go:216] Machine: {Timestamp:2026-06-11 13:10:37.10953678 +0000 UTC m=+0.352614630 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092502 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26aeb0f8a0de311c5740ab7bc3e4d5 SystemUUID:ec26aeb0-f8a0-de31-1c57-40ab7bc3e4d5 BootID:5c05de1c-edc3-44ef-8ad3-c2e8c035d5be Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:97:45:b8:19:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:97:45:b8:19:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:d5:42:64:36:89 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 13:10:37.110996 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.110991 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 13:10:37.111094 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.111065 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 13:10:37.113996 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.113969 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 13:10:37.114126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.113999 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-252.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 13:10:37.114659 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.114650 2563 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 13:10:37.114689 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.114662 2563 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 13:10:37.114689 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.114670 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 13:10:37.115384 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.115374 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 13:10:37.116214 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.116203 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 13:10:37.119212 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119201 2563 state_mem.go:36] "Initialized new in-memory state store" Jun 11 13:10:37.119318 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119308 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 13:10:37.119358 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119337 2563 kubelet.go:486] "Attempting to sync node with API server" Jun 11 13:10:37.119358 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119347 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 13:10:37.119424 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119360 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 13:10:37.119424 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119370 2563 kubelet.go:398] "Adding apiserver pod source" Jun 11 13:10:37.119424 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119377 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 13:10:37.119735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119723 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 13:10:37.119768 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.119741 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 13:10:37.121855 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.121838 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 13:10:37.123221 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.123207 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 13:10:37.123285 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.123238 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 13:10:37.124307 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124295 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 13:10:37.124353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124311 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 13:10:37.124353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124318 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 13:10:37.124353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124323 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 13:10:37.124353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124332 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 13:10:37.124353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124337 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 13:10:37.124353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124346 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 13:10:37.124353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124351 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 13:10:37.124524 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124358 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 13:10:37.124524 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124364 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 13:10:37.124524 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124380 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 13:10:37.124524 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.124385 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 13:10:37.125373 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.125364 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 13:10:37.125373 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.125373 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 13:10:37.127904 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.127886 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sd9lq" Jun 11 13:10:37.128534 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.128520 2563 server.go:1268] "Started kubelet" Jun 11 13:10:37.128645 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.128608 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 13:10:37.128707 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.128612 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 13:10:37.128758 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.128725 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 13:10:37.129325 ip-10-0-141-252 systemd[1]: Started Kubernetes Kubelet. Jun 11 13:10:37.133369 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.133341 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 13:10:37.134646 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.134607 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sd9lq" Jun 11 13:10:37.135563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.135541 2563 server.go:310] "Adding debug handlers to kubelet server" Jun 11 13:10:37.136832 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.136793 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 11 13:10:37.137495 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.137479 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-252.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 13:10:37.137712 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.137690 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-252.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 13:10:37.137791 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.137772 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 13:10:37.138588 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.137501 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-252.ec2.internal.18b808e621da5871 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-252.ec2.internal,UID:ip-10-0-141-252.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-252.ec2.internal,},FirstTimestamp:2026-06-11 13:10:37.128489073 +0000 UTC m=+0.371566897,LastTimestamp:2026-06-11 13:10:37.128489073 +0000 UTC m=+0.371566897,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-252.ec2.internal,}" Jun 11 13:10:37.140810 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.140792 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 13:10:37.140890 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.140810 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 13:10:37.140943 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.140893 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.141501 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.141481 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 13:10:37.141583 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.141526 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 13:10:37.141680 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.141665 2563 reconstruct.go:97] "Volume reconstruction finished" Jun 11 13:10:37.141680 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.141679 2563 reconciler.go:29] "Reconciler: start to sync state" Jun 11 13:10:37.141796 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.141752 2563 factory.go:55] Registering systemd factory Jun 11 13:10:37.141796 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.141792 2563 factory.go:223] Registration of the systemd container factory successfully Jun 11 13:10:37.141891 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.141846 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 13:10:37.142036 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.142025 2563 factory.go:153] Registering CRI-O factory Jun 11 13:10:37.142070 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.142040 2563 factory.go:223] Registration of the crio container factory successfully Jun 11 13:10:37.142102 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.142084 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 13:10:37.142133 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.142102 2563 factory.go:103] Registering Raw factory Jun 11 13:10:37.142133 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.142112 2563 manager.go:1196] Started watching for new ooms in manager Jun 11 13:10:37.142573 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.142563 2563 manager.go:349] Starting recovery of all containers Jun 11 13:10:37.143198 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.143175 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 11 13:10:37.143271 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.143250 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-252.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 11 13:10:37.153868 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.153855 2563 manager.go:354] Recovery completed Jun 11 13:10:37.158388 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.158374 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 13:10:37.160043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.160029 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientMemory" Jun 11 13:10:37.160129 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.160067 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 13:10:37.160129 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.160081 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientPID" Jun 11 13:10:37.160787 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.160774 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 13:10:37.160787 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.160782 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 13:10:37.160787 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.160787 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 13:10:37.160948 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.160806 2563 state_mem.go:36] "Initialized new in-memory state store" Jun 11 13:10:37.161567 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.161551 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientMemory" Jun 11 13:10:37.161661 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.161578 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 13:10:37.161661 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.161592 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientPID" Jun 11 13:10:37.162207 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.162196 2563 policy_none.go:49] "None policy: Start" Jun 11 13:10:37.162248 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.162212 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 13:10:37.162248 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.162222 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 13:10:37.163295 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.163285 2563 policy_none.go:47] "Start" Jun 11 13:10:37.197248 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.197228 2563 manager.go:337] "Starting Device Plugin manager" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.197269 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.197284 2563 server.go:87] "Starting device plugin registration server" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.197528 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.197538 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.197633 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.197723 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.197732 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.199502 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.199551 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.209895 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.209826 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 13:10:37.211118 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.211097 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 13:10:37.211188 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.211151 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 13:10:37.211188 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.211175 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 13:10:37.211264 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.211215 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 11 13:10:37.213379 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.213358 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 13:10:37.298632 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.298601 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 13:10:37.305542 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.305494 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientMemory" Jun 11 13:10:37.305542 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.305526 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 13:10:37.305542 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.305540 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientPID" Jun 11 13:10:37.305695 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.305565 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.311661 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.311638 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal"] Jun 11 13:10:37.311749 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.311702 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 13:10:37.316590 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.316574 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientMemory" Jun 11 13:10:37.316678 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.316609 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 13:10:37.316678 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.316641 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientPID" Jun 11 13:10:37.317257 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.317244 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.317290 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.317265 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-252.ec2.internal\": node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.317920 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.317908 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 13:10:37.318043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.318028 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.318081 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.318076 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 13:10:37.318574 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.318557 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientMemory" Jun 11 13:10:37.318668 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.318579 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 13:10:37.318668 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.318589 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientPID" Jun 11 13:10:37.318668 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.318615 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientMemory" Jun 11 13:10:37.318668 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.318654 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 13:10:37.318668 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.318668 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientPID" Jun 11 13:10:37.319914 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.319901 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.319974 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.319925 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 13:10:37.320540 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.320525 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientMemory" Jun 11 13:10:37.320616 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.320545 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 13:10:37.320616 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.320554 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeHasSufficientPID" Jun 11 13:10:37.332972 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.332956 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.336162 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.336149 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-252.ec2.internal\" not found" node="ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.339962 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.339948 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-252.ec2.internal\" not found" node="ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.343123 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.343096 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b732421dcc2171bb17036784d182879-config\") pod \"kube-apiserver-proxy-ip-10-0-141-252.ec2.internal\" (UID: \"5b732421dcc2171bb17036784d182879\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.343123 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.343122 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aedacd29ac3815faa218c4f623501e15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal\" (UID: \"aedacd29ac3815faa218c4f623501e15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.343238 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.343138 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aedacd29ac3815faa218c4f623501e15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal\" (UID: \"aedacd29ac3815faa218c4f623501e15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.433931 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.433899 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.443419 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.443394 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aedacd29ac3815faa218c4f623501e15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal\" (UID: \"aedacd29ac3815faa218c4f623501e15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.443500 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.443425 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aedacd29ac3815faa218c4f623501e15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal\" (UID: \"aedacd29ac3815faa218c4f623501e15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.443500 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.443441 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b732421dcc2171bb17036784d182879-config\") pod \"kube-apiserver-proxy-ip-10-0-141-252.ec2.internal\" (UID: \"5b732421dcc2171bb17036784d182879\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.443500 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.443480 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b732421dcc2171bb17036784d182879-config\") pod \"kube-apiserver-proxy-ip-10-0-141-252.ec2.internal\" (UID: \"5b732421dcc2171bb17036784d182879\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.443598 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.443508 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aedacd29ac3815faa218c4f623501e15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal\" (UID: \"aedacd29ac3815faa218c4f623501e15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.443598 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.443506 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aedacd29ac3815faa218c4f623501e15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal\" (UID: \"aedacd29ac3815faa218c4f623501e15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.533992 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.533961 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.634915 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.634847 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.639982 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.639969 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.644756 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:37.644233 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" Jun 11 13:10:37.735316 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.735283 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.835881 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.835850 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:37.936350 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:37.936321 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:38.037038 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:38.037010 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:38.073567 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.073540 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 13:10:38.074147 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.073712 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 13:10:38.129298 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:38.129263 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b732421dcc2171bb17036784d182879.slice/crio-ace8b56e827fcb31c8fbe498b3a2f8fa65d360adaa1ab62def182f2b46e9bc2a WatchSource:0}: Error finding container ace8b56e827fcb31c8fbe498b3a2f8fa65d360adaa1ab62def182f2b46e9bc2a: Status 404 returned error can't find the container with id ace8b56e827fcb31c8fbe498b3a2f8fa65d360adaa1ab62def182f2b46e9bc2a Jun 11 13:10:38.129663 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:38.129638 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaedacd29ac3815faa218c4f623501e15.slice/crio-4d28dcc39dc825cecc5001bf3f305a76eed5d24b8ff3836772db444c0bd6a3e8 WatchSource:0}: Error finding container 4d28dcc39dc825cecc5001bf3f305a76eed5d24b8ff3836772db444c0bd6a3e8: Status 404 returned error can't find the container with id 4d28dcc39dc825cecc5001bf3f305a76eed5d24b8ff3836772db444c0bd6a3e8 Jun 11 13:10:38.133763 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.133744 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 13:10:38.137840 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:38.137822 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:38.139314 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.139284 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 13:05:37 +0000 UTC" deadline="2028-03-17 16:02:23.678975192 +0000 UTC" Jun 11 13:10:38.139378 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.139315 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15482h51m45.539663966s" Jun 11 13:10:38.217538 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.217429 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" event={"ID":"5b732421dcc2171bb17036784d182879","Type":"ContainerStarted","Data":"ace8b56e827fcb31c8fbe498b3a2f8fa65d360adaa1ab62def182f2b46e9bc2a"} Jun 11 13:10:38.218461 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.218442 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" event={"ID":"aedacd29ac3815faa218c4f623501e15","Type":"ContainerStarted","Data":"4d28dcc39dc825cecc5001bf3f305a76eed5d24b8ff3836772db444c0bd6a3e8"} Jun 11 13:10:38.238592 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:38.238569 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-252.ec2.internal\" not found" Jun 11 13:10:38.300815 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.300787 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 13:10:38.341848 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.341829 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" Jun 11 13:10:38.349586 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.349567 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 13:10:38.351324 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.351312 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" Jun 11 13:10:38.360523 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.360508 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 13:10:38.657890 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.657862 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 13:10:38.663963 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:38.663927 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 13:10:39.120651 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.120556 2563 apiserver.go:52] "Watching apiserver" Jun 11 13:10:39.130068 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.130039 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 13:10:39.130552 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.130524 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6xs65","kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal","openshift-multus/multus-mtgr8","openshift-network-operator/iptables-alerter-q8h5l","openshift-ovn-kubernetes/ovnkube-node-pxgg2","kube-system/konnectivity-agent-p5hp9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc","openshift-cluster-node-tuning-operator/tuned-vhb2d","openshift-dns/node-resolver-5msxz","openshift-image-registry/node-ca-qqwrv","openshift-multus/multus-additional-cni-plugins-sgntk","openshift-multus/network-metrics-daemon-lwnxx","openshift-network-diagnostics/network-check-target-rw5vd"] Jun 11 13:10:39.132312 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.132293 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:10:39.133461 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.133437 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:39.133559 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.133523 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:39.134659 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.134606 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:39.134751 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.134696 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:39.134798 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.134750 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 13:10:39.134798 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.134789 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cn5wl\"" Jun 11 13:10:39.134879 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.134794 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 13:10:39.136133 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.136114 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.137418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.137397 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.138738 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.138667 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 13:10:39.138738 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.138712 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 13:10:39.138875 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.138763 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jzgml\"" Jun 11 13:10:39.138875 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.138669 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 13:10:39.138875 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.138667 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 13:10:39.138875 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.138833 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.140164 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.139908 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-skttm\"" Jun 11 13:10:39.140164 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.139921 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 13:10:39.140164 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.139969 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 13:10:39.140164 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.139984 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 13:10:39.140376 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.140364 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.141883 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.142261 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.142281 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nvw2k\"" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.142397 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.142484 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.142526 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.142598 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.142667 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 13:10:39.142800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.142720 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 13:10:39.143386 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.143363 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.143474 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.143458 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ljl7g\"" Jun 11 13:10:39.143529 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.143459 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:39.143659 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.143640 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 13:10:39.147116 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.147098 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.148509 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.148488 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.149832 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.149814 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 13:10:39.150012 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.149816 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 13:10:39.150099 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.150013 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.150292 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.149842 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4xfcn\"" Jun 11 13:10:39.150336 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.149825 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 13:10:39.151287 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.151266 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9xdts\"" Jun 11 13:10:39.151391 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.151369 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 13:10:39.151505 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.151487 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 13:10:39.151836 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.151815 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.152239 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.152223 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 13:10:39.152377 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.152363 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vxgjs\"" Jun 11 13:10:39.152848 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.152832 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 13:10:39.152999 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.152839 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 13:10:39.153294 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153276 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysconfig\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.153371 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153308 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c5689cf-8303-455c-b659-5d3f9714d416-konnectivity-ca\") pod \"konnectivity-agent-p5hp9\" (UID: \"7c5689cf-8303-455c-b659-5d3f9714d416\") " pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:10:39.153371 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-daemon-config\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.153371 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153357 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-cni-bin\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.153531 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153403 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46db2288-b49b-45b5-a190-2f8946b9ae70-ovn-node-metrics-cert\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.153531 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-run\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.153531 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-slash\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.153680 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-kubernetes\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.153680 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysctl-d\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.153680 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153580 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-lib-modules\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.153680 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34af1083-79db-4cbb-b331-3e9c5bac1b1d-host-slash\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.153858 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153681 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xss4d\" (UniqueName: \"kubernetes.io/projected/34af1083-79db-4cbb-b331-3e9c5bac1b1d-kube-api-access-xss4d\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.153858 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153732 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-systemd\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.153858 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153776 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.153858 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153822 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-systemd\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.153858 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-cni-multus\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153872 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-conf-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153906 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fddd497e-da5a-4f4f-adcf-7041387c2d92-kubelet-config\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153929 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fddd497e-da5a-4f4f-adcf-7041387c2d92-dbus\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153955 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-cni-bin\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.153978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-multus-certs\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-kubelet\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154024 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-sys\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154052 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-var-lib-kubelet\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.154096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-os-release\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/34af1083-79db-4cbb-b331-3e9c5bac1b1d-iptables-alerter-script\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154133 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-systemd-units\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-env-overrides\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154177 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a32166a-f652-4a18-8c92-5c38f0868a36-cni-binary-copy\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154199 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-netns\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154221 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-var-lib-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154245 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-ovnkube-config\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154276 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c5689cf-8303-455c-b659-5d3f9714d416-agent-certs\") pod \"konnectivity-agent-p5hp9\" (UID: \"7c5689cf-8303-455c-b659-5d3f9714d416\") " pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154289 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-socket-dir-parent\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154303 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-cni-netd\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154337 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-ovnkube-script-lib\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-host\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8h8h\" (UniqueName: \"kubernetes.io/projected/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-kube-api-access-r8h8h\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-cnibin\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.154446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154435 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7l4q\" (UniqueName: \"kubernetes.io/projected/46db2288-b49b-45b5-a190-2f8946b9ae70-kube-api-access-t7l4q\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-tuned\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154484 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysctl-conf\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154508 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-cni-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154531 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-kubelet\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154553 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4fx8\" (UniqueName: \"kubernetes.io/projected/0a32166a-f652-4a18-8c92-5c38f0868a36-kube-api-access-l4fx8\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154575 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-etc-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154652 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154682 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154675 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-system-cni-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154716 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-hostroot\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154751 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-k8s-cni-cncf-io\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154781 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-run-netns\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-log-socket\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154840 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-run-ovn-kubernetes\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154853 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-modprobe-d\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.155146 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-ovn\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.155761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-node-log\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.155761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154910 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqkkb\" (UniqueName: \"kubernetes.io/projected/9364f758-a646-4d6c-902a-9e361a572682-kube-api-access-bqkkb\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:39.155761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154932 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-tmp\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.155761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154948 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.155761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.154972 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-etc-kubernetes\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.155761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.155068 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l468q\"" Jun 11 13:10:39.155761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.155169 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 13:10:39.172891 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.172873 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7ghqf" Jun 11 13:10:39.182883 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.182862 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7ghqf" Jun 11 13:10:39.243263 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.243235 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 13:10:39.255971 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.255935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysctl-conf\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.256126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.255984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-cni-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-kubelet\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4fx8\" (UniqueName: \"kubernetes.io/projected/0a32166a-f652-4a18-8c92-5c38f0868a36-kube-api-access-l4fx8\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256060 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncln8\" (UniqueName: \"kubernetes.io/projected/e4985a94-028c-4cc0-a1ec-78a753a8f0db-kube-api-access-ncln8\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.256126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256085 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.256126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-etc-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-system-cni-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256181 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-hostroot\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-socket-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256214 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-cni-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256228 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-cnibin\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256215 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-kubelet\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256260 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-etc-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256279 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-hostroot\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-k8s-cni-cncf-io\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.256303 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-sys-fs\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256329 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-system-cni-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.256368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256369 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-k8s-cni-cncf-io\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.256375 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs podName:9364f758-a646-4d6c-902a-9e361a572682 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:39.756357919 +0000 UTC m=+2.999435731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs") pod "network-metrics-daemon-lwnxx" (UID: "9364f758-a646-4d6c-902a-9e361a572682") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256398 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysctl-conf\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256430 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256476 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90114819-e819-4921-9de2-a9260098a343-host\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256527 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-run-netns\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-log-socket\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256596 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-run-netns\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-run-ovn-kubernetes\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256615 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-log-socket\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256657 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-modprobe-d\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256692 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e504aab2-b809-4e56-8f01-57513a7af2b6-tmp-dir\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256750 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-run-ovn-kubernetes\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-ovn\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-node-log\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256816 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-modprobe-d\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkkb\" (UniqueName: \"kubernetes.io/projected/9364f758-a646-4d6c-902a-9e361a572682-kube-api-access-bqkkb\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256869 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-node-log\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256880 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-ovn\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256911 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-tmp\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.256933 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.257046 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.257111 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret podName:fddd497e-da5a-4f4f-adcf-7041387c2d92 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:39.757093553 +0000 UTC m=+3.000171374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret") pod "global-pull-secret-syncer-6xs65" (UID: "fddd497e-da5a-4f4f-adcf-7041387c2d92") : object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-etc-selinux\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-cni-binary-copy\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-etc-kubernetes\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257245 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-registration-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysconfig\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257293 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c5689cf-8303-455c-b659-5d3f9714d416-konnectivity-ca\") pod \"konnectivity-agent-p5hp9\" (UID: \"7c5689cf-8303-455c-b659-5d3f9714d416\") " pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257349 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysconfig\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257381 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-daemon-config\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257408 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.257722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257432 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-system-cni-dir\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-cni-bin\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46db2288-b49b-45b5-a190-2f8946b9ae70-ovn-node-metrics-cert\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257541 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-run\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257550 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-cni-bin\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257542 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257572 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-slash\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257678 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-kubernetes\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysctl-d\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257745 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-lib-modules\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34af1083-79db-4cbb-b331-3e9c5bac1b1d-host-slash\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257797 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xss4d\" (UniqueName: \"kubernetes.io/projected/34af1083-79db-4cbb-b331-3e9c5bac1b1d-kube-api-access-xss4d\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257821 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/90114819-e819-4921-9de2-a9260098a343-serviceca\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257847 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-systemd\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.257969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-systemd\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-cni-multus\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-conf-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.258421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fddd497e-da5a-4f4f-adcf-7041387c2d92-kubelet-config\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258090 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fddd497e-da5a-4f4f-adcf-7041387c2d92-dbus\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258120 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-cni-bin\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258134 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-daemon-config\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258154 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-multus-certs\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258179 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-kubelet\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258215 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-sys\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258242 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-var-lib-kubelet\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258271 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-os-release\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258300 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/34af1083-79db-4cbb-b331-3e9c5bac1b1d-iptables-alerter-script\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-systemd-units\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258496 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-env-overrides\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a32166a-f652-4a18-8c92-5c38f0868a36-cni-binary-copy\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258549 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-netns\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258579 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79xpb\" (UniqueName: \"kubernetes.io/projected/e504aab2-b809-4e56-8f01-57513a7af2b6-kube-api-access-79xpb\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-os-release\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-var-lib-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258685 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259175 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-ovnkube-config\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c5689cf-8303-455c-b659-5d3f9714d416-agent-certs\") pod \"konnectivity-agent-p5hp9\" (UID: \"7c5689cf-8303-455c-b659-5d3f9714d416\") " pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-socket-dir-parent\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258808 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-device-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e504aab2-b809-4e56-8f01-57513a7af2b6-hosts-file\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-cni-netd\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-ovnkube-script-lib\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-host\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.258957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8h8h\" (UniqueName: \"kubernetes.io/projected/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-kube-api-access-r8h8h\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.259025 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-cnibin\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.259040 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/34af1083-79db-4cbb-b331-3e9c5bac1b1d-iptables-alerter-script\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.259057 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtsmj\" (UniqueName: \"kubernetes.io/projected/90114819-e819-4921-9de2-a9260098a343-kube-api-access-dtsmj\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.259087 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7l4q\" (UniqueName: \"kubernetes.io/projected/46db2288-b49b-45b5-a190-2f8946b9ae70-kube-api-access-t7l4q\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.259118 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-slash\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.259129 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-tuned\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.259087 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-systemd\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.259961 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.259706 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trbz2\" (UniqueName: \"kubernetes.io/projected/02f763af-9de2-4dbe-91b0-929a96b0ef17-kube-api-access-trbz2\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260095 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-run-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260113 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-cnibin\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260154 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-host\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260174 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-conf-dir\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260168 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260194 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fddd497e-da5a-4f4f-adcf-7041387c2d92-kubelet-config\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260252 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-kubernetes\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260253 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-multus-certs\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260303 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-run\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260353 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fddd497e-da5a-4f4f-adcf-7041387c2d92-dbus\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260390 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-cni-bin\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260402 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-sysctl-d\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260411 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-kubelet\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260439 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-etc-kubernetes\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260517 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34af1083-79db-4cbb-b331-3e9c5bac1b1d-host-slash\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260647 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-env-overrides\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260652 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-lib-modules\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.261563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260763 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c5689cf-8303-455c-b659-5d3f9714d416-konnectivity-ca\") pod \"konnectivity-agent-p5hp9\" (UID: \"7c5689cf-8303-455c-b659-5d3f9714d416\") " pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260791 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-run-netns\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260815 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-ovnkube-config\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260822 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-systemd-units\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260847 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-os-release\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260881 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-var-lib-kubelet\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260892 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-host-var-lib-cni-multus\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260889 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-sys\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260943 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0a32166a-f652-4a18-8c92-5c38f0868a36-multus-socket-dir-parent\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260952 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-host-cni-netd\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260954 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-systemd\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.260995 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46db2288-b49b-45b5-a190-2f8946b9ae70-var-lib-openvswitch\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.261123 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a32166a-f652-4a18-8c92-5c38f0868a36-cni-binary-copy\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.262241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.261418 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46db2288-b49b-45b5-a190-2f8946b9ae70-ovnkube-script-lib\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.263699 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.263546 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 13:10:39.263699 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.263573 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 13:10:39.263699 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.263588 2563 projected.go:196] Error preparing data for projected volume kube-api-access-6sg2b for pod openshift-network-diagnostics/network-check-target-rw5vd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:39.263699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.263594 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46db2288-b49b-45b5-a190-2f8946b9ae70-ovn-node-metrics-cert\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.263699 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.263671 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b podName:34a4f037-4b0b-4ffc-8e07-3e47dee50718 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:39.763654348 +0000 UTC m=+3.006732181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6sg2b" (UniqueName: "kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b") pod "network-check-target-rw5vd" (UID: "34a4f037-4b0b-4ffc-8e07-3e47dee50718") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:39.264042 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.263739 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c5689cf-8303-455c-b659-5d3f9714d416-agent-certs\") pod \"konnectivity-agent-p5hp9\" (UID: \"7c5689cf-8303-455c-b659-5d3f9714d416\") " pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:10:39.264256 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.264200 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-etc-tuned\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.266323 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.266289 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkkb\" (UniqueName: \"kubernetes.io/projected/9364f758-a646-4d6c-902a-9e361a572682-kube-api-access-bqkkb\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:39.266634 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.266567 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4fx8\" (UniqueName: \"kubernetes.io/projected/0a32166a-f652-4a18-8c92-5c38f0868a36-kube-api-access-l4fx8\") pod \"multus-mtgr8\" (UID: \"0a32166a-f652-4a18-8c92-5c38f0868a36\") " pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.267671 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.267653 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8h8h\" (UniqueName: \"kubernetes.io/projected/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-kube-api-access-r8h8h\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.268952 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.268927 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xss4d\" (UniqueName: \"kubernetes.io/projected/34af1083-79db-4cbb-b331-3e9c5bac1b1d-kube-api-access-xss4d\") pod \"iptables-alerter-q8h5l\" (UID: \"34af1083-79db-4cbb-b331-3e9c5bac1b1d\") " pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.269533 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.269516 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7l4q\" (UniqueName: \"kubernetes.io/projected/46db2288-b49b-45b5-a190-2f8946b9ae70-kube-api-access-t7l4q\") pod \"ovnkube-node-pxgg2\" (UID: \"46db2288-b49b-45b5-a190-2f8946b9ae70\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.270008 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.269992 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd7eba43-af9c-4b78-8cc6-f453ae5ca632-tmp\") pod \"tuned-vhb2d\" (UID: \"cd7eba43-af9c-4b78-8cc6-f453ae5ca632\") " pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.360306 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360273 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79xpb\" (UniqueName: \"kubernetes.io/projected/e504aab2-b809-4e56-8f01-57513a7af2b6-kube-api-access-79xpb\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.360306 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360305 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-os-release\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.360594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360331 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-device-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.360594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e504aab2-b809-4e56-8f01-57513a7af2b6-hosts-file\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.360594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360384 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtsmj\" (UniqueName: \"kubernetes.io/projected/90114819-e819-4921-9de2-a9260098a343-kube-api-access-dtsmj\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.360594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trbz2\" (UniqueName: \"kubernetes.io/projected/02f763af-9de2-4dbe-91b0-929a96b0ef17-kube-api-access-trbz2\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.360594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360420 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-os-release\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.360594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360436 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncln8\" (UniqueName: \"kubernetes.io/projected/e4985a94-028c-4cc0-a1ec-78a753a8f0db-kube-api-access-ncln8\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.360594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360426 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-device-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.360594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360652 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-socket-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360681 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e504aab2-b809-4e56-8f01-57513a7af2b6-hosts-file\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-cnibin\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360737 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-cnibin\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360783 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-sys-fs\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360810 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-socket-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360838 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90114819-e819-4921-9de2-a9260098a343-host\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.360882 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360868 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-sys-fs\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.361132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360900 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90114819-e819-4921-9de2-a9260098a343-host\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.361132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360872 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e504aab2-b809-4e56-8f01-57513a7af2b6-tmp-dir\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.361132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360927 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.361132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360954 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-etc-selinux\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.361132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360966 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.361132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361067 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.361132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361127 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e504aab2-b809-4e56-8f01-57513a7af2b6-tmp-dir\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.360971 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-cni-binary-copy\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361171 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-registration-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361179 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-etc-selinux\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-system-cni-dir\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361244 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-registration-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361269 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/90114819-e819-4921-9de2-a9260098a343-serviceca\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361290 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4985a94-028c-4cc0-a1ec-78a753a8f0db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.361329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361305 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.361563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361359 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02f763af-9de2-4dbe-91b0-929a96b0ef17-system-cni-dir\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.361605 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.361589 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/90114819-e819-4921-9de2-a9260098a343-serviceca\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.362316 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.362290 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02f763af-9de2-4dbe-91b0-929a96b0ef17-cni-binary-copy\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.371246 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.371188 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trbz2\" (UniqueName: \"kubernetes.io/projected/02f763af-9de2-4dbe-91b0-929a96b0ef17-kube-api-access-trbz2\") pod \"multus-additional-cni-plugins-sgntk\" (UID: \"02f763af-9de2-4dbe-91b0-929a96b0ef17\") " pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.371792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.371726 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79xpb\" (UniqueName: \"kubernetes.io/projected/e504aab2-b809-4e56-8f01-57513a7af2b6-kube-api-access-79xpb\") pod \"node-resolver-5msxz\" (UID: \"e504aab2-b809-4e56-8f01-57513a7af2b6\") " pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.371792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.371754 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtsmj\" (UniqueName: \"kubernetes.io/projected/90114819-e819-4921-9de2-a9260098a343-kube-api-access-dtsmj\") pod \"node-ca-qqwrv\" (UID: \"90114819-e819-4921-9de2-a9260098a343\") " pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.376254 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.376226 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncln8\" (UniqueName: \"kubernetes.io/projected/e4985a94-028c-4cc0-a1ec-78a753a8f0db-kube-api-access-ncln8\") pod \"aws-ebs-csi-driver-node-dc2dc\" (UID: \"e4985a94-028c-4cc0-a1ec-78a753a8f0db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.430016 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.429985 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 13:10:39.445332 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.445306 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:10:39.452904 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.452882 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mtgr8" Jun 11 13:10:39.461360 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.461338 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q8h5l" Jun 11 13:10:39.466744 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.466724 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:10:39.474277 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.474243 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" Jun 11 13:10:39.481870 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.481850 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" Jun 11 13:10:39.488830 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.488810 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5msxz" Jun 11 13:10:39.496191 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.496158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qqwrv" Jun 11 13:10:39.501414 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.501392 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sgntk" Jun 11 13:10:39.765494 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.765464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:39.765671 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.765503 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:39.765671 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:39.765532 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:39.765671 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.765638 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:39.765671 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.765662 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 13:10:39.765883 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.765682 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 13:10:39.765883 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.765689 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:39.765883 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.765697 2563 projected.go:196] Error preparing data for projected volume kube-api-access-6sg2b for pod openshift-network-diagnostics/network-check-target-rw5vd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:39.765883 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.765709 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs podName:9364f758-a646-4d6c-902a-9e361a572682 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:40.765688231 +0000 UTC m=+4.008766047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs") pod "network-metrics-daemon-lwnxx" (UID: "9364f758-a646-4d6c-902a-9e361a572682") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:39.765883 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.765735 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret podName:fddd497e-da5a-4f4f-adcf-7041387c2d92 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:40.765721679 +0000 UTC m=+4.008799491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret") pod "global-pull-secret-syncer-6xs65" (UID: "fddd497e-da5a-4f4f-adcf-7041387c2d92") : object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:39.765883 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:39.765754 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b podName:34a4f037-4b0b-4ffc-8e07-3e47dee50718 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:40.765745191 +0000 UTC m=+4.008823003 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sg2b" (UniqueName: "kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b") pod "network-check-target-rw5vd" (UID: "34a4f037-4b0b-4ffc-8e07-3e47dee50718") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:40.115037 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:40.115011 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a32166a_f652_4a18_8c92_5c38f0868a36.slice/crio-604b601ea4116fda5a50e199ee9ee390716d7c2f73f1d3f8502d3c421045b811 WatchSource:0}: Error finding container 604b601ea4116fda5a50e199ee9ee390716d7c2f73f1d3f8502d3c421045b811: Status 404 returned error can't find the container with id 604b601ea4116fda5a50e199ee9ee390716d7c2f73f1d3f8502d3c421045b811 Jun 11 13:10:40.116670 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:40.116643 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34af1083_79db_4cbb_b331_3e9c5bac1b1d.slice/crio-fbb47591de4a5b42f9b59422562a93c677ca85c6c6b40b725b75375e066824c3 WatchSource:0}: Error finding container fbb47591de4a5b42f9b59422562a93c677ca85c6c6b40b725b75375e066824c3: Status 404 returned error can't find the container with id fbb47591de4a5b42f9b59422562a93c677ca85c6c6b40b725b75375e066824c3 Jun 11 13:10:40.117379 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:40.117290 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7eba43_af9c_4b78_8cc6_f453ae5ca632.slice/crio-1070a8198a9a2d23055d3020833f77da65ecdc603a22095cc73b06165ebc8f45 WatchSource:0}: Error finding container 1070a8198a9a2d23055d3020833f77da65ecdc603a22095cc73b06165ebc8f45: Status 404 returned error can't find the container with id 1070a8198a9a2d23055d3020833f77da65ecdc603a22095cc73b06165ebc8f45 Jun 11 13:10:40.119492 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:40.119469 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f763af_9de2_4dbe_91b0_929a96b0ef17.slice/crio-0415074fdf4bb4dad3fae93c954460370cc429eb280c16fd5a55b3ca4fee3b31 WatchSource:0}: Error finding container 0415074fdf4bb4dad3fae93c954460370cc429eb280c16fd5a55b3ca4fee3b31: Status 404 returned error can't find the container with id 0415074fdf4bb4dad3fae93c954460370cc429eb280c16fd5a55b3ca4fee3b31 Jun 11 13:10:40.121246 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:40.121226 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90114819_e819_4921_9de2_a9260098a343.slice/crio-77df66bb516f44c2fd72d8875c2c0420705ecac9732ed8bbba2795ca8f6a1ce0 WatchSource:0}: Error finding container 77df66bb516f44c2fd72d8875c2c0420705ecac9732ed8bbba2795ca8f6a1ce0: Status 404 returned error can't find the container with id 77df66bb516f44c2fd72d8875c2c0420705ecac9732ed8bbba2795ca8f6a1ce0 Jun 11 13:10:40.122706 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:10:40.122603 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5689cf_8303_455c_b659_5d3f9714d416.slice/crio-0699c09a67187f85afc7ee3fc0d87df6692f5351d2467ae8e59458588a65ad6a WatchSource:0}: Error finding container 0699c09a67187f85afc7ee3fc0d87df6692f5351d2467ae8e59458588a65ad6a: Status 404 returned error can't find the container with id 0699c09a67187f85afc7ee3fc0d87df6692f5351d2467ae8e59458588a65ad6a Jun 11 13:10:40.186301 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.186259 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 13:05:39 +0000 UTC" deadline="2027-12-12 08:42:41.3530438 +0000 UTC" Jun 11 13:10:40.186301 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.186300 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13171h32m1.166745843s" Jun 11 13:10:40.223781 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.223742 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"ec38be00af3787b6aae5634c388a0dd0bedff4dc85312a7fa38acaf28884a5f6"} Jun 11 13:10:40.224747 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.224722 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" event={"ID":"e4985a94-028c-4cc0-a1ec-78a753a8f0db","Type":"ContainerStarted","Data":"53d72c4adc85fd91288f1fffbb7462eab91964e6a40bce7619b4e4aac673f5ac"} Jun 11 13:10:40.225749 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.225724 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5msxz" event={"ID":"e504aab2-b809-4e56-8f01-57513a7af2b6","Type":"ContainerStarted","Data":"72d8aeb1f403263775668c8fb8cb82c7ef2bcbed4006606a0ae1f7ae23b1ed31"} Jun 11 13:10:40.229088 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.229060 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p5hp9" event={"ID":"7c5689cf-8303-455c-b659-5d3f9714d416","Type":"ContainerStarted","Data":"0699c09a67187f85afc7ee3fc0d87df6692f5351d2467ae8e59458588a65ad6a"} Jun 11 13:10:40.233880 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.233856 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qqwrv" event={"ID":"90114819-e819-4921-9de2-a9260098a343","Type":"ContainerStarted","Data":"77df66bb516f44c2fd72d8875c2c0420705ecac9732ed8bbba2795ca8f6a1ce0"} Jun 11 13:10:40.238037 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.238011 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerStarted","Data":"0415074fdf4bb4dad3fae93c954460370cc429eb280c16fd5a55b3ca4fee3b31"} Jun 11 13:10:40.239058 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.239007 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mtgr8" event={"ID":"0a32166a-f652-4a18-8c92-5c38f0868a36","Type":"ContainerStarted","Data":"604b601ea4116fda5a50e199ee9ee390716d7c2f73f1d3f8502d3c421045b811"} Jun 11 13:10:40.243005 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.242985 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" event={"ID":"5b732421dcc2171bb17036784d182879","Type":"ContainerStarted","Data":"1bab341694c04f1c48feb43ad5388c4beacfb5888169150953840efe287ff30f"} Jun 11 13:10:40.243996 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.243972 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" event={"ID":"cd7eba43-af9c-4b78-8cc6-f453ae5ca632","Type":"ContainerStarted","Data":"1070a8198a9a2d23055d3020833f77da65ecdc603a22095cc73b06165ebc8f45"} Jun 11 13:10:40.245012 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.244979 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q8h5l" event={"ID":"34af1083-79db-4cbb-b331-3e9c5bac1b1d","Type":"ContainerStarted","Data":"fbb47591de4a5b42f9b59422562a93c677ca85c6c6b40b725b75375e066824c3"} Jun 11 13:10:40.258509 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.258464 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-252.ec2.internal" podStartSLOduration=2.258452644 podStartE2EDuration="2.258452644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:10:40.258095297 +0000 UTC m=+3.501173129" watchObservedRunningTime="2026-06-11 13:10:40.258452644 +0000 UTC m=+3.501530474" Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.773615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.773689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:40.773731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:40.773878 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:40.773941 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret podName:fddd497e-da5a-4f4f-adcf-7041387c2d92 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:42.773923091 +0000 UTC m=+6.017000906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret") pod "global-pull-secret-syncer-6xs65" (UID: "fddd497e-da5a-4f4f-adcf-7041387c2d92") : object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:40.774343 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:40.774398 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs podName:9364f758-a646-4d6c-902a-9e361a572682 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:42.774381957 +0000 UTC m=+6.017459773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs") pod "network-metrics-daemon-lwnxx" (UID: "9364f758-a646-4d6c-902a-9e361a572682") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:40.774476 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:40.774489 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:40.774501 2563 projected.go:196] Error preparing data for projected volume kube-api-access-6sg2b for pod openshift-network-diagnostics/network-check-target-rw5vd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:40.774578 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:40.774538 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b podName:34a4f037-4b0b-4ffc-8e07-3e47dee50718 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:42.774526424 +0000 UTC m=+6.017604254 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sg2b" (UniqueName: "kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b") pod "network-check-target-rw5vd" (UID: "34a4f037-4b0b-4ffc-8e07-3e47dee50718") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:41.188844 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:41.188726 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 13:05:39 +0000 UTC" deadline="2028-01-22 09:20:09.120014002 +0000 UTC" Jun 11 13:10:41.188844 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:41.188771 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14156h9m27.931256714s" Jun 11 13:10:41.213449 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:41.212733 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:41.213449 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:41.212872 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:41.213449 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:41.213281 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:41.213449 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:41.213388 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:41.213955 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:41.213830 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:41.213955 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:41.213920 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:41.261644 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:41.261395 2563 generic.go:356] "Generic (PLEG): container finished" podID="aedacd29ac3815faa218c4f623501e15" containerID="ca6e4a9aa2a5dac043cc79344fad07afa7a3f1448d0dd9b13cccb373a6c449c9" exitCode=0 Jun 11 13:10:41.261881 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:41.261827 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" event={"ID":"aedacd29ac3815faa218c4f623501e15","Type":"ContainerDied","Data":"ca6e4a9aa2a5dac043cc79344fad07afa7a3f1448d0dd9b13cccb373a6c449c9"} Jun 11 13:10:42.283213 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:42.283173 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" event={"ID":"aedacd29ac3815faa218c4f623501e15","Type":"ContainerStarted","Data":"47a99a8ff0ccdf4ad31eddff43d3207dbbe0633050990a218c296a0fcafca59f"} Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:42.796155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:42.796212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:42.796250 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:42.796369 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:42.796427 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret podName:fddd497e-da5a-4f4f-adcf-7041387c2d92 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:46.796409969 +0000 UTC m=+10.039487786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret") pod "global-pull-secret-syncer-6xs65" (UID: "fddd497e-da5a-4f4f-adcf-7041387c2d92") : object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:42.796830 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:42.796884 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs podName:9364f758-a646-4d6c-902a-9e361a572682 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:46.796866008 +0000 UTC m=+10.039943825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs") pod "network-metrics-daemon-lwnxx" (UID: "9364f758-a646-4d6c-902a-9e361a572682") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:42.796964 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:42.796981 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:42.797003 2563 projected.go:196] Error preparing data for projected volume kube-api-access-6sg2b for pod openshift-network-diagnostics/network-check-target-rw5vd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:42.797089 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:42.797037 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b podName:34a4f037-4b0b-4ffc-8e07-3e47dee50718 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:46.797026082 +0000 UTC m=+10.040103907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sg2b" (UniqueName: "kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b") pod "network-check-target-rw5vd" (UID: "34a4f037-4b0b-4ffc-8e07-3e47dee50718") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:43.211932 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:43.211404 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:43.211932 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:43.211514 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:43.211932 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:43.211691 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:43.211932 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:43.211774 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:43.211932 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:43.211825 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:43.211932 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:43.211898 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:45.215711 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:45.215678 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:45.215711 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:45.215701 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:45.216249 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:45.215688 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:45.216249 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:45.215824 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:45.216249 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:45.215929 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:45.216249 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:45.216036 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:46.832962 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:46.832921 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:46.832976 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:46.833019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:46.833073 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:46.833131 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:46.833139 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs podName:9364f758-a646-4d6c-902a-9e361a572682 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:54.83311869 +0000 UTC m=+18.076196521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs") pod "network-metrics-daemon-lwnxx" (UID: "9364f758-a646-4d6c-902a-9e361a572682") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:46.833164 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret podName:fddd497e-da5a-4f4f-adcf-7041387c2d92 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:54.833154943 +0000 UTC m=+18.076232759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret") pod "global-pull-secret-syncer-6xs65" (UID: "fddd497e-da5a-4f4f-adcf-7041387c2d92") : object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:46.833238 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:46.833252 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:46.833264 2563 projected.go:196] Error preparing data for projected volume kube-api-access-6sg2b for pod openshift-network-diagnostics/network-check-target-rw5vd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:46.833353 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:46.833299 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b podName:34a4f037-4b0b-4ffc-8e07-3e47dee50718 nodeName:}" failed. No retries permitted until 2026-06-11 13:10:54.833287943 +0000 UTC m=+18.076365772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sg2b" (UniqueName: "kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b") pod "network-check-target-rw5vd" (UID: "34a4f037-4b0b-4ffc-8e07-3e47dee50718") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:47.216081 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:47.216037 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:47.216281 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:47.216215 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:47.216853 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:47.216833 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:47.217032 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:47.217002 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:47.217267 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:47.217010 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:47.218143 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:47.218070 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:49.211925 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:49.211879 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:49.211925 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:49.211923 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:49.212472 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:49.212012 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:49.212472 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:49.211879 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:49.212472 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:49.212148 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:49.212472 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:49.212226 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:51.211999 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:51.211966 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:51.212459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:51.211965 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:51.212459 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:51.212098 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:51.212459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:51.211970 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:51.212459 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:51.212165 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:51.212459 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:51.212225 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:53.214788 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:53.214759 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:53.215207 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:53.214759 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:53.215207 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:53.214882 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:53.215207 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:53.214864 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:53.215207 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:53.214987 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:53.215207 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:53.215049 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:54.893257 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:54.893169 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:54.893257 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:54.893213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:54.893257 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:54.893244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:54.893797 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:54.893356 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:54.893797 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:54.893358 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 13:10:54.893797 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:54.893395 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 13:10:54.893797 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:54.893411 2563 projected.go:196] Error preparing data for projected volume kube-api-access-6sg2b for pod openshift-network-diagnostics/network-check-target-rw5vd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:54.893797 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:54.893421 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret podName:fddd497e-da5a-4f4f-adcf-7041387c2d92 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:10.89340255 +0000 UTC m=+34.136480368 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret") pod "global-pull-secret-syncer-6xs65" (UID: "fddd497e-da5a-4f4f-adcf-7041387c2d92") : object "kube-system"/"original-pull-secret" not registered Jun 11 13:10:54.893797 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:54.893438 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:54.893797 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:54.893484 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b podName:34a4f037-4b0b-4ffc-8e07-3e47dee50718 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:10.893441551 +0000 UTC m=+34.136519362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sg2b" (UniqueName: "kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b") pod "network-check-target-rw5vd" (UID: "34a4f037-4b0b-4ffc-8e07-3e47dee50718") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:10:54.893797 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:54.893518 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs podName:9364f758-a646-4d6c-902a-9e361a572682 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:10.89350442 +0000 UTC m=+34.136582231 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs") pod "network-metrics-daemon-lwnxx" (UID: "9364f758-a646-4d6c-902a-9e361a572682") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:10:55.211829 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:55.211796 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:55.212029 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:55.211801 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:55.212029 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:55.211943 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:55.212029 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:55.212002 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:55.212029 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:55.211801 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:55.212240 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:55.212092 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:57.212675 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:57.212639 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:57.213171 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:57.212754 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:57.213171 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:57.213144 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:57.213309 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:57.213230 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:57.213309 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:57.213287 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:57.213412 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:57.213358 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:59.215479 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.215279 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:10:59.215989 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:59.215587 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:10:59.215989 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.215331 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:10:59.215989 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:59.215711 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:10:59.215989 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.215285 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:10:59.215989 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:10:59.215813 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:10:59.330489 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.329745 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qqwrv" event={"ID":"90114819-e819-4921-9de2-a9260098a343","Type":"ContainerStarted","Data":"79056b22fe3ea1123c13cb7919e587a70217597067f64400a21e2f3a8f5a7565"} Jun 11 13:10:59.333684 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.333294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerStarted","Data":"a74d822f4bd2158f5108da36fc35d3214726f6c5336bfe27821a840800ac6826"} Jun 11 13:10:59.336064 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.336042 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mtgr8" event={"ID":"0a32166a-f652-4a18-8c92-5c38f0868a36","Type":"ContainerStarted","Data":"d84872badf02bf64a6ecfe2dd700eab3075abba51b51f19cf904f0666fb99364"} Jun 11 13:10:59.337589 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.337568 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" event={"ID":"cd7eba43-af9c-4b78-8cc6-f453ae5ca632","Type":"ContainerStarted","Data":"ff191dc3f48f75091a80db0efc1d07a838e1162508e95ce08cc5365004728274"} Jun 11 13:10:59.339196 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.339177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"634953f2744d968f1a4b3b7708babd0e3dbabf58f991ff0a4d66b671638381e1"} Jun 11 13:10:59.340662 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.340644 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" event={"ID":"e4985a94-028c-4cc0-a1ec-78a753a8f0db","Type":"ContainerStarted","Data":"e60a1db0896be4f53c73f4b78bd7d5623e742cc0998d8638f20b87a7951823a1"} Jun 11 13:10:59.342154 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.342134 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5msxz" event={"ID":"e504aab2-b809-4e56-8f01-57513a7af2b6","Type":"ContainerStarted","Data":"f9afdf17a3eccb12a7c01d43bffe7dd01d0c5827e547709ce534e110601b280a"} Jun 11 13:10:59.343597 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.343579 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p5hp9" event={"ID":"7c5689cf-8303-455c-b659-5d3f9714d416","Type":"ContainerStarted","Data":"ad56740c768ac3a2ccdae98bda94ea4520516559dbb90e91e95a60ca0d1666e8"} Jun 11 13:10:59.344249 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.344202 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-252.ec2.internal" podStartSLOduration=21.344189096 podStartE2EDuration="21.344189096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:10:42.297865461 +0000 UTC m=+5.540943295" watchObservedRunningTime="2026-06-11 13:10:59.344189096 +0000 UTC m=+22.587266930" Jun 11 13:10:59.344905 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.344817 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qqwrv" podStartSLOduration=3.548089568 podStartE2EDuration="22.344807694s" totalImagesPullingTime="18.796718126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:10:59.344111209 +0000 UTC m=+22.587189042" watchObservedRunningTime="2026-06-11 13:10:59.344807694 +0000 UTC m=+22.587885528" Jun 11 13:10:59.359125 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.359088 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p5hp9" podStartSLOduration=3.575029132 podStartE2EDuration="22.359075856s" totalImagesPullingTime="18.784046724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:10:59.358796654 +0000 UTC m=+22.601874507" watchObservedRunningTime="2026-06-11 13:10:59.359075856 +0000 UTC m=+22.602153688" Jun 11 13:10:59.377602 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.377568 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mtgr8" podStartSLOduration=3.548215551 podStartE2EDuration="22.377555362s" totalImagesPullingTime="18.829339811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:10:59.377319904 +0000 UTC m=+22.620397737" watchObservedRunningTime="2026-06-11 13:10:59.377555362 +0000 UTC m=+22.620633195" Jun 11 13:10:59.395497 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.395457 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vhb2d" podStartSLOduration=3.594837108 podStartE2EDuration="22.395444217s" totalImagesPullingTime="18.800607109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:10:59.395153786 +0000 UTC m=+22.638231619" watchObservedRunningTime="2026-06-11 13:10:59.395444217 +0000 UTC m=+22.638522049" Jun 11 13:10:59.413130 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:10:59.413094 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5msxz" podStartSLOduration=3.630999864 podStartE2EDuration="22.41308166s" totalImagesPullingTime="18.782081796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:10:59.412812637 +0000 UTC m=+22.655890469" watchObservedRunningTime="2026-06-11 13:10:59.41308166 +0000 UTC m=+22.656159493" Jun 11 13:11:00.347825 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:00.347791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"f1b5a12b1d176167fd9121b5e52bf7cc172331e713b745aae0057610a5f0dc3a"} Jun 11 13:11:00.348554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:00.347903 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"8139f7db36183dd414dd68a58582f8c584c27e5e80ae01faec352a90dfad57e4"} Jun 11 13:11:00.348554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:00.347947 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"29e845c9a0dd0f508bb84b59c87948702dd23e5da9cf986148a884b8d05fd73a"} Jun 11 13:11:00.348554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:00.347995 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"a298339ab9bbecbafe005ade094f056097a787d0c922e1afa57a686fb77217de"} Jun 11 13:11:00.348554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:00.348040 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"b1bcaadb3c4dd72eac08ee87b2931299244bec8ebe75c082a2de8be3e49821b8"} Jun 11 13:11:00.349234 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:00.349211 2563 generic.go:356] "Generic (PLEG): container finished" podID="02f763af-9de2-4dbe-91b0-929a96b0ef17" containerID="a74d822f4bd2158f5108da36fc35d3214726f6c5336bfe27821a840800ac6826" exitCode=0 Jun 11 13:11:00.349327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:00.349306 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerDied","Data":"a74d822f4bd2158f5108da36fc35d3214726f6c5336bfe27821a840800ac6826"} Jun 11 13:11:00.748066 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:00.748038 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 13:11:01.210647 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.210495 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T13:11:00.748061064Z","UUID":"84760266-6adc-4bbf-a969-6c0aa37b4251","Handler":null,"Name":"","Endpoint":""} Jun 11 13:11:01.213181 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.213151 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:01.217800 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:01.217729 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:11:01.217955 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.217862 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:01.217955 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.217902 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:01.218065 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.217976 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 13:11:01.218065 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.218027 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 13:11:01.218167 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:01.218082 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:11:01.218846 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:01.218251 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:11:01.354099 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.353691 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q8h5l" event={"ID":"34af1083-79db-4cbb-b331-3e9c5bac1b1d","Type":"ContainerStarted","Data":"b3ad7f2ec11876b8d959abf79b88a0fa28d47171161e46de56054e489d599359"} Jun 11 13:11:01.355723 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.355682 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" event={"ID":"e4985a94-028c-4cc0-a1ec-78a753a8f0db","Type":"ContainerStarted","Data":"50e1680fff1c28629bbf038de027a8cb5bf24b012643dcf7ef9df6b021f44c61"} Jun 11 13:11:01.368964 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:01.368920 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q8h5l" podStartSLOduration=5.562759742 podStartE2EDuration="24.368906862s" totalImagesPullingTime="18.80614712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:01.368718591 +0000 UTC m=+24.611796451" watchObservedRunningTime="2026-06-11 13:11:01.368906862 +0000 UTC m=+24.611984695" Jun 11 13:11:02.147148 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:02.146922 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:11:02.147822 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:02.147612 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:11:02.362377 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:02.362290 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"6938701904bea4df05b880e67a929079118fd8ac13cf1c2a5a8748e8995ae412"} Jun 11 13:11:02.364777 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:02.364752 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" event={"ID":"e4985a94-028c-4cc0-a1ec-78a753a8f0db","Type":"ContainerStarted","Data":"25d3c1b45421790448fed38493992020a34cee0b2b4fdd12ba1918e9c05d1b80"} Jun 11 13:11:02.365022 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:02.365002 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:11:02.365522 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:02.365506 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p5hp9" Jun 11 13:11:02.383586 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:02.383544 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dc2dc" podStartSLOduration=4.460651023 podStartE2EDuration="25.383530955s" totalImagesPullingTime="20.922879932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:02.382792373 +0000 UTC m=+25.625870208" watchObservedRunningTime="2026-06-11 13:11:02.383530955 +0000 UTC m=+25.626608789" Jun 11 13:11:03.212022 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:03.211976 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:03.212022 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:03.212016 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:03.212276 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:03.212128 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:11:03.212585 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:03.212562 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:11:03.212641 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:03.212610 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:03.212727 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:03.212706 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:11:04.374508 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:04.374344 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" event={"ID":"46db2288-b49b-45b5-a190-2f8946b9ae70","Type":"ContainerStarted","Data":"aaea75b8f5662f4557b68871e59f1c45a1a4a7f43e01f74ac6a5830d5774eec7"} Jun 11 13:11:04.403393 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:04.403260 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" podStartSLOduration=8.302658021 podStartE2EDuration="27.403238147s" totalImagesPullingTime="19.100580126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:04.402769033 +0000 UTC m=+27.645846865" watchObservedRunningTime="2026-06-11 13:11:04.403238147 +0000 UTC m=+27.646315982" Jun 11 13:11:05.212152 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.212122 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:05.212316 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.212122 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:05.212316 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:05.212213 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:11:05.212316 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.212134 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:05.212471 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:05.212323 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:11:05.212471 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:05.212379 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:11:05.381670 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.381637 2563 generic.go:356] "Generic (PLEG): container finished" podID="02f763af-9de2-4dbe-91b0-929a96b0ef17" containerID="c33991800d61ba369fbbd0aa47c78f90939a1f35a9f9a4ce1e4f0095fc9ed31e" exitCode=0 Jun 11 13:11:05.382050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.381700 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerDied","Data":"c33991800d61ba369fbbd0aa47c78f90939a1f35a9f9a4ce1e4f0095fc9ed31e"} Jun 11 13:11:05.382050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.381767 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 13:11:05.382329 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.382309 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:11:05.382454 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.382404 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:11:05.398812 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.398789 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:11:05.398914 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:05.398882 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:11:06.225731 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.225698 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lwnxx"] Jun 11 13:11:06.225877 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.225835 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:06.225972 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:06.225949 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:11:06.246807 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.246714 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rw5vd"] Jun 11 13:11:06.246949 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.246832 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:06.246949 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:06.246932 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:11:06.247392 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.247369 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6xs65"] Jun 11 13:11:06.247483 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.247466 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:06.247567 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:06.247542 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:11:06.386556 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.386524 2563 generic.go:356] "Generic (PLEG): container finished" podID="02f763af-9de2-4dbe-91b0-929a96b0ef17" containerID="8f3207424b97a74dde9d99c3fb910b66d8fb96e593f822d4721a059b9ab45362" exitCode=0 Jun 11 13:11:06.386949 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.386598 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerDied","Data":"8f3207424b97a74dde9d99c3fb910b66d8fb96e593f822d4721a059b9ab45362"} Jun 11 13:11:06.386949 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:06.386722 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 13:11:07.394552 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:07.394325 2563 generic.go:356] "Generic (PLEG): container finished" podID="02f763af-9de2-4dbe-91b0-929a96b0ef17" containerID="d10ddb6965880e6029c142721e429dc83b3653762657af3d1c61b6b72b1e3683" exitCode=0 Jun 11 13:11:07.394552 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:07.394391 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerDied","Data":"d10ddb6965880e6029c142721e429dc83b3653762657af3d1c61b6b72b1e3683"} Jun 11 13:11:07.394932 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:07.394608 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 13:11:08.212141 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:08.212111 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:08.212309 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:08.212229 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:08.212309 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:08.212235 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:11:08.212309 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:08.212240 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:08.212433 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:08.212404 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:11:08.212512 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:08.212496 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:11:09.303412 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:09.303378 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:11:09.304074 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:09.303603 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 13:11:09.315377 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:09.315341 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" podUID="46db2288-b49b-45b5-a190-2f8946b9ae70" containerName="ovnkube-controller" probeResult="failure" output="" Jun 11 13:11:09.324915 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:09.324881 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" podUID="46db2288-b49b-45b5-a190-2f8946b9ae70" containerName="ovnkube-controller" probeResult="failure" output="" Jun 11 13:11:10.211583 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:10.211555 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:10.211583 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:10.211571 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:10.211848 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:10.211596 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:10.211848 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.211699 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:11:10.211953 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.211840 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:11:10.211953 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.211939 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:11:10.910418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:10.910383 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:10.910418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:10.910422 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:10.910459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.910530 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.910547 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.910571 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.910575 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.910586 2563 projected.go:196] Error preparing data for projected volume kube-api-access-6sg2b for pod openshift-network-diagnostics/network-check-target-rw5vd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.910599 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs podName:9364f758-a646-4d6c-902a-9e361a572682 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:42.910582023 +0000 UTC m=+66.153659837 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs") pod "network-metrics-daemon-lwnxx" (UID: "9364f758-a646-4d6c-902a-9e361a572682") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.910642 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret podName:fddd497e-da5a-4f4f-adcf-7041387c2d92 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:42.910609255 +0000 UTC m=+66.153687088 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret") pod "global-pull-secret-syncer-6xs65" (UID: "fddd497e-da5a-4f4f-adcf-7041387c2d92") : object "kube-system"/"original-pull-secret" not registered Jun 11 13:11:10.911082 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:10.910660 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b podName:34a4f037-4b0b-4ffc-8e07-3e47dee50718 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:42.910650856 +0000 UTC m=+66.153728667 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sg2b" (UniqueName: "kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b") pod "network-check-target-rw5vd" (UID: "34a4f037-4b0b-4ffc-8e07-3e47dee50718") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 13:11:12.211783 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:12.211760 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:12.212347 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:12.211760 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:12.212347 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:12.211895 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwnxx" podUID="9364f758-a646-4d6c-902a-9e361a572682" Jun 11 13:11:12.212347 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:12.211760 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:12.212347 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:12.211942 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6xs65" podUID="fddd497e-da5a-4f4f-adcf-7041387c2d92" Jun 11 13:11:12.212347 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:12.212005 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rw5vd" podUID="34a4f037-4b0b-4ffc-8e07-3e47dee50718" Jun 11 13:11:12.407649 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:12.407601 2563 generic.go:356] "Generic (PLEG): container finished" podID="02f763af-9de2-4dbe-91b0-929a96b0ef17" containerID="d77de72ba03db8c650688de558c77a04c6a421ebcfd0bd230c72e0e8428a4a08" exitCode=0 Jun 11 13:11:12.407649 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:12.407642 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerDied","Data":"d77de72ba03db8c650688de558c77a04c6a421ebcfd0bd230c72e0e8428a4a08"} Jun 11 13:11:12.572785 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:12.572751 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:12.648277 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:12.648234 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:13.029597 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.029572 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-252.ec2.internal" event="NodeReady" Jun 11 13:11:13.029789 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.029716 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 13:11:13.059703 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.059677 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76c675c55d-cqxtm"] Jun 11 13:11:13.080942 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.080742 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6bbwn"] Jun 11 13:11:13.081748 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.081725 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.084355 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.084333 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9w7l7\"" Jun 11 13:11:13.084355 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.084344 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 11 13:11:13.084730 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.084711 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 11 13:11:13.084828 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.084776 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 11 13:11:13.089755 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.089687 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 11 13:11:13.106188 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.106171 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t294z"] Jun 11 13:11:13.106304 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.106288 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.108849 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.108819 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 13:11:13.108849 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.108830 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s9zw4\"" Jun 11 13:11:13.109006 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.108881 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 13:11:13.124082 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.124059 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76c675c55d-cqxtm"] Jun 11 13:11:13.124082 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.124085 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6bbwn"] Jun 11 13:11:13.124203 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.124101 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t294z"] Jun 11 13:11:13.124203 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.124122 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t294z" Jun 11 13:11:13.126726 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.126707 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 13:11:13.126842 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.126824 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 13:11:13.126902 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.126833 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 13:11:13.126942 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.126932 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q8tj7\"" Jun 11 13:11:13.227517 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227490 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/811a9938-4e3f-45fc-8c18-a7e91fc04dc8-cert\") pod \"ingress-canary-t294z\" (UID: \"811a9938-4e3f-45fc-8c18-a7e91fc04dc8\") " pod="openshift-ingress-canary/ingress-canary-t294z" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227536 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-installation-pull-secrets\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-trusted-ca\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/602cc0fa-d5b5-49db-bd64-cb8c85b52291-ca-trust-extracted\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227650 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-certificates\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227681 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4dh6\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-kube-api-access-g4dh6\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227713 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-image-registry-private-configuration\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227741 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9cnq\" (UniqueName: \"kubernetes.io/projected/811a9938-4e3f-45fc-8c18-a7e91fc04dc8-kube-api-access-g9cnq\") pod \"ingress-canary-t294z\" (UID: \"811a9938-4e3f-45fc-8c18-a7e91fc04dc8\") " pod="openshift-ingress-canary/ingress-canary-t294z" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d7fa167-12b1-4e91-93e6-0722528075d2-tmp-dir\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227818 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227849 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nbv4\" (UniqueName: \"kubernetes.io/projected/2d7fa167-12b1-4e91-93e6-0722528075d2-kube-api-access-5nbv4\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.227930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-bound-sa-token\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.228258 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227951 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d7fa167-12b1-4e91-93e6-0722528075d2-config-volume\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.228258 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.227969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.328847 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.328780 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-trusted-ca\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.328847 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.328812 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/602cc0fa-d5b5-49db-bd64-cb8c85b52291-ca-trust-extracted\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.328847 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.328832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-certificates\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.329060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.328859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4dh6\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-kube-api-access-g4dh6\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.329060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.328884 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-image-registry-private-configuration\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.329060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.328913 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9cnq\" (UniqueName: \"kubernetes.io/projected/811a9938-4e3f-45fc-8c18-a7e91fc04dc8-kube-api-access-g9cnq\") pod \"ingress-canary-t294z\" (UID: \"811a9938-4e3f-45fc-8c18-a7e91fc04dc8\") " pod="openshift-ingress-canary/ingress-canary-t294z" Jun 11 13:11:13.329060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.328944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d7fa167-12b1-4e91-93e6-0722528075d2-tmp-dir\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.329060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.328978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.329060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nbv4\" (UniqueName: \"kubernetes.io/projected/2d7fa167-12b1-4e91-93e6-0722528075d2-kube-api-access-5nbv4\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.329060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-bound-sa-token\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d7fa167-12b1-4e91-93e6-0722528075d2-config-volume\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/811a9938-4e3f-45fc-8c18-a7e91fc04dc8-cert\") pod \"ingress-canary-t294z\" (UID: \"811a9938-4e3f-45fc-8c18-a7e91fc04dc8\") " pod="openshift-ingress-canary/ingress-canary-t294z" Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329174 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-installation-pull-secrets\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329244 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/602cc0fa-d5b5-49db-bd64-cb8c85b52291-ca-trust-extracted\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.329283 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.329297 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c675c55d-cqxtm: secret "image-registry-tls" not found Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.329326 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.329358 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls podName:602cc0fa-d5b5-49db-bd64-cb8c85b52291 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:13.829339303 +0000 UTC m=+37.072417113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls") pod "image-registry-76c675c55d-cqxtm" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291") : secret "image-registry-tls" not found Jun 11 13:11:13.329400 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.329375 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls podName:2d7fa167-12b1-4e91-93e6-0722528075d2 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:13.829367257 +0000 UTC m=+37.072445071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls") pod "dns-default-6bbwn" (UID: "2d7fa167-12b1-4e91-93e6-0722528075d2") : secret "dns-default-metrics-tls" not found Jun 11 13:11:13.330117 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329486 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d7fa167-12b1-4e91-93e6-0722528075d2-tmp-dir\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.330117 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329888 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d7fa167-12b1-4e91-93e6-0722528075d2-config-volume\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.330117 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.329967 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-certificates\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.330275 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.330253 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-trusted-ca\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.333281 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.333255 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/811a9938-4e3f-45fc-8c18-a7e91fc04dc8-cert\") pod \"ingress-canary-t294z\" (UID: \"811a9938-4e3f-45fc-8c18-a7e91fc04dc8\") " pod="openshift-ingress-canary/ingress-canary-t294z" Jun 11 13:11:13.333281 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.333270 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-installation-pull-secrets\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.333399 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.333258 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-image-registry-private-configuration\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.339014 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.338968 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9cnq\" (UniqueName: \"kubernetes.io/projected/811a9938-4e3f-45fc-8c18-a7e91fc04dc8-kube-api-access-g9cnq\") pod \"ingress-canary-t294z\" (UID: \"811a9938-4e3f-45fc-8c18-a7e91fc04dc8\") " pod="openshift-ingress-canary/ingress-canary-t294z" Jun 11 13:11:13.339192 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.339171 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4dh6\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-kube-api-access-g4dh6\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.339275 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.339228 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nbv4\" (UniqueName: \"kubernetes.io/projected/2d7fa167-12b1-4e91-93e6-0722528075d2-kube-api-access-5nbv4\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.339347 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.339328 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-bound-sa-token\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.412354 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.412329 2563 generic.go:356] "Generic (PLEG): container finished" podID="02f763af-9de2-4dbe-91b0-929a96b0ef17" containerID="67ce21ff842a2ce5e9520e4eacb07da5ed5ad4263b3b274404210c3c65e06245" exitCode=0 Jun 11 13:11:13.412435 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.412363 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerDied","Data":"67ce21ff842a2ce5e9520e4eacb07da5ed5ad4263b3b274404210c3c65e06245"} Jun 11 13:11:13.431761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.431643 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t294z" Jun 11 13:11:13.602849 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.602777 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t294z"] Jun 11 13:11:13.605996 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:13.605969 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811a9938_4e3f_45fc_8c18_a7e91fc04dc8.slice/crio-0efa9aa0101cb1be857c5f3ea1a47d430bc5fb989587795df880b087582539b5 WatchSource:0}: Error finding container 0efa9aa0101cb1be857c5f3ea1a47d430bc5fb989587795df880b087582539b5: Status 404 returned error can't find the container with id 0efa9aa0101cb1be857c5f3ea1a47d430bc5fb989587795df880b087582539b5 Jun 11 13:11:13.833281 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.833252 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:13.833419 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:13.833303 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:13.833419 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.833386 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 13:11:13.833419 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.833391 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 13:11:13.833419 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.833408 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c675c55d-cqxtm: secret "image-registry-tls" not found Jun 11 13:11:13.833543 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.833431 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls podName:2d7fa167-12b1-4e91-93e6-0722528075d2 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:14.833418565 +0000 UTC m=+38.076496375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls") pod "dns-default-6bbwn" (UID: "2d7fa167-12b1-4e91-93e6-0722528075d2") : secret "dns-default-metrics-tls" not found Jun 11 13:11:13.833543 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:13.833451 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls podName:602cc0fa-d5b5-49db-bd64-cb8c85b52291 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:14.833436615 +0000 UTC m=+38.076514453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls") pod "image-registry-76c675c55d-cqxtm" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291") : secret "image-registry-tls" not found Jun 11 13:11:14.211949 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.211911 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:14.212109 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.212004 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:14.212510 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.212241 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:14.214986 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.214965 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 13:11:14.215084 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.214991 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 13:11:14.215084 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.214994 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 13:11:14.215084 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.214965 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lb4sh\"" Jun 11 13:11:14.216134 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.216115 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 13:11:14.216215 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.216118 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2fc6r\"" Jun 11 13:11:14.416399 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.416367 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t294z" event={"ID":"811a9938-4e3f-45fc-8c18-a7e91fc04dc8","Type":"ContainerStarted","Data":"0efa9aa0101cb1be857c5f3ea1a47d430bc5fb989587795df880b087582539b5"} Jun 11 13:11:14.420226 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.420200 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgntk" event={"ID":"02f763af-9de2-4dbe-91b0-929a96b0ef17","Type":"ContainerStarted","Data":"6592046ef1333567edd5b6882b95e2445677c8efabfe3736495e25fd128c2b15"} Jun 11 13:11:14.441707 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.441652 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sgntk" podStartSLOduration=9.084248773 podStartE2EDuration="37.441616582s" totalImagesPullingTime="28.357367809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:14.439505392 +0000 UTC m=+37.682583252" watchObservedRunningTime="2026-06-11 13:11:14.441616582 +0000 UTC m=+37.684694417" Jun 11 13:11:14.559303 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.559230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:14.618824 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.618783 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:14.842616 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.842379 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:14.842616 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:14.842563 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 13:11:14.842616 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:14.842587 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c675c55d-cqxtm: secret "image-registry-tls" not found Jun 11 13:11:14.842898 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:14.842615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:14.842898 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:14.842668 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls podName:602cc0fa-d5b5-49db-bd64-cb8c85b52291 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:16.842648222 +0000 UTC m=+40.085726052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls") pod "image-registry-76c675c55d-cqxtm" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291") : secret "image-registry-tls" not found Jun 11 13:11:14.842898 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:14.842725 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 13:11:14.842898 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:14.842785 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls podName:2d7fa167-12b1-4e91-93e6-0722528075d2 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:16.84276826 +0000 UTC m=+40.085846074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls") pod "dns-default-6bbwn" (UID: "2d7fa167-12b1-4e91-93e6-0722528075d2") : secret "dns-default-metrics-tls" not found Jun 11 13:11:15.513995 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:15.513958 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:15.579693 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:15.579653 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:16.426095 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:16.426044 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t294z" event={"ID":"811a9938-4e3f-45fc-8c18-a7e91fc04dc8","Type":"ContainerStarted","Data":"d58e17891289a60cbb44c5df75a71c4f6da3136c3b519217a9a4f5cac498a7d4"} Jun 11 13:11:16.442166 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:16.442123 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t294z" podStartSLOduration=0.915666315 podStartE2EDuration="3.442108812s" totalImagesPullingTime="2.526442497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:16.440761962 +0000 UTC m=+39.683839794" watchObservedRunningTime="2026-06-11 13:11:16.442108812 +0000 UTC m=+39.685186644" Jun 11 13:11:16.854718 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:16.854652 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:16.854718 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:16.854704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:16.855177 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:16.854796 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 13:11:16.855177 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:16.854799 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 13:11:16.855177 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:16.854816 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c675c55d-cqxtm: secret "image-registry-tls" not found Jun 11 13:11:16.855177 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:16.854854 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls podName:602cc0fa-d5b5-49db-bd64-cb8c85b52291 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:20.854838145 +0000 UTC m=+44.097915957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls") pod "image-registry-76c675c55d-cqxtm" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291") : secret "image-registry-tls" not found Jun 11 13:11:16.855177 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:16.854870 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls podName:2d7fa167-12b1-4e91-93e6-0722528075d2 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:20.854861742 +0000 UTC m=+44.097939552 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls") pod "dns-default-6bbwn" (UID: "2d7fa167-12b1-4e91-93e6-0722528075d2") : secret "dns-default-metrics-tls" not found Jun 11 13:11:17.006045 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:17.006006 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:17.056271 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:17.056235 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:18.115889 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.115846 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:18.165103 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.165066 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:18.616705 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.616668 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:18.632528 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.632498 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n"] Jun 11 13:11:18.662352 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.662319 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n"] Jun 11 13:11:18.662461 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.662413 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" Jun 11 13:11:18.665132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.665101 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jpzpz\"" Jun 11 13:11:18.665132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.665114 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 11 13:11:18.666108 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.666092 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 11 13:11:18.684718 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.684689 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:18.753239 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.753209 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:18.768183 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.768156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxbpx\" (UniqueName: \"kubernetes.io/projected/e72d2172-73fa-4df4-8f86-8bd19ce68941-kube-api-access-dxbpx\") pod \"migrator-955ffdf5f-h8r8n\" (UID: \"e72d2172-73fa-4df4-8f86-8bd19ce68941\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" Jun 11 13:11:18.853330 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.853298 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:18.869344 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.869293 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxbpx\" (UniqueName: \"kubernetes.io/projected/e72d2172-73fa-4df4-8f86-8bd19ce68941-kube-api-access-dxbpx\") pod \"migrator-955ffdf5f-h8r8n\" (UID: \"e72d2172-73fa-4df4-8f86-8bd19ce68941\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" Jun 11 13:11:18.880836 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.880814 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxbpx\" (UniqueName: \"kubernetes.io/projected/e72d2172-73fa-4df4-8f86-8bd19ce68941-kube-api-access-dxbpx\") pod \"migrator-955ffdf5f-h8r8n\" (UID: \"e72d2172-73fa-4df4-8f86-8bd19ce68941\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" Jun 11 13:11:18.904991 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.904967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:18.970944 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:18.970920 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" Jun 11 13:11:19.122937 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:19.122864 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n"] Jun 11 13:11:19.125783 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:19.125752 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode72d2172_73fa_4df4_8f86_8bd19ce68941.slice/crio-a8793d31713ad1965a4c464cdaff8a81ccf7f52d24d4649648ee3b0f9e219aeb WatchSource:0}: Error finding container a8793d31713ad1965a4c464cdaff8a81ccf7f52d24d4649648ee3b0f9e219aeb: Status 404 returned error can't find the container with id a8793d31713ad1965a4c464cdaff8a81ccf7f52d24d4649648ee3b0f9e219aeb Jun 11 13:11:19.238243 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:19.238205 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:19.290885 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:19.290854 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:19.426689 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:19.426667 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5msxz_e504aab2-b809-4e56-8f01-57513a7af2b6/dns-node-resolver/0.log" Jun 11 13:11:19.433370 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:19.433339 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" event={"ID":"e72d2172-73fa-4df4-8f86-8bd19ce68941","Type":"ContainerStarted","Data":"a8793d31713ad1965a4c464cdaff8a81ccf7f52d24d4649648ee3b0f9e219aeb"} Jun 11 13:11:19.513335 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:19.513302 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:19.614757 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:19.614723 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:20.426497 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:20.426465 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qqwrv_90114819-e819-4921-9de2-a9260098a343/node-ca/0.log" Jun 11 13:11:20.826796 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:20.826721 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t294z_811a9938-4e3f-45fc-8c18-a7e91fc04dc8/serve-healthcheck-canary/0.log" Jun 11 13:11:20.884169 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:20.884134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:20.884316 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:20.884222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:20.884316 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:20.884276 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 13:11:20.884437 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:20.884333 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls podName:2d7fa167-12b1-4e91-93e6-0722528075d2 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:28.88431814 +0000 UTC m=+52.127395951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls") pod "dns-default-6bbwn" (UID: "2d7fa167-12b1-4e91-93e6-0722528075d2") : secret "dns-default-metrics-tls" not found Jun 11 13:11:20.884437 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:20.884351 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 13:11:20.884437 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:20.884366 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c675c55d-cqxtm: secret "image-registry-tls" not found Jun 11 13:11:20.884437 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:20.884422 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls podName:602cc0fa-d5b5-49db-bd64-cb8c85b52291 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:28.884405887 +0000 UTC m=+52.127483712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls") pod "image-registry-76c675c55d-cqxtm" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291") : secret "image-registry-tls" not found Jun 11 13:11:21.110052 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.109980 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-gxp8r"] Jun 11 13:11:21.125211 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.125189 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-gxp8r"] Jun 11 13:11:21.125326 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.125272 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.127948 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.127915 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 11 13:11:21.127948 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.127939 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 11 13:11:21.127948 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.127945 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 11 13:11:21.128147 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.127966 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 11 13:11:21.129208 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.129186 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vlxdn\"" Jun 11 13:11:21.151521 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.151500 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:21.199658 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.199614 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:21.287436 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.287410 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2681592e-0c00-42e9-947d-a313fb719677-signing-cabundle\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.287527 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.287468 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh5cv\" (UniqueName: \"kubernetes.io/projected/2681592e-0c00-42e9-947d-a313fb719677-kube-api-access-kh5cv\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.287563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.287524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2681592e-0c00-42e9-947d-a313fb719677-signing-key\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.388593 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.388570 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2681592e-0c00-42e9-947d-a313fb719677-signing-cabundle\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.388700 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.388655 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh5cv\" (UniqueName: \"kubernetes.io/projected/2681592e-0c00-42e9-947d-a313fb719677-kube-api-access-kh5cv\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.388764 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.388699 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2681592e-0c00-42e9-947d-a313fb719677-signing-key\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.389128 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.389109 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2681592e-0c00-42e9-947d-a313fb719677-signing-cabundle\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.390956 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.390939 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2681592e-0c00-42e9-947d-a313fb719677-signing-key\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.396831 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.396808 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh5cv\" (UniqueName: \"kubernetes.io/projected/2681592e-0c00-42e9-947d-a313fb719677-kube-api-access-kh5cv\") pod \"service-ca-7d74d578b8-gxp8r\" (UID: \"2681592e-0c00-42e9-947d-a313fb719677\") " pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.433818 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.433801 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" Jun 11 13:11:21.439247 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.439228 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" event={"ID":"e72d2172-73fa-4df4-8f86-8bd19ce68941","Type":"ContainerStarted","Data":"f92e3ecb82de5a3537e74bcf5650304dbcecffd1f3ed3148bd414d5131d819b7"} Jun 11 13:11:21.439331 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.439270 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" event={"ID":"e72d2172-73fa-4df4-8f86-8bd19ce68941","Type":"ContainerStarted","Data":"001ebdbbc963615cf7dc62a2e12ea8586f5fdffe506dd43f43d9207ed7c4da10"} Jun 11 13:11:21.453667 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.453608 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-h8r8n" podStartSLOduration=1.434143728 podStartE2EDuration="3.453596841s" totalImagesPullingTime="2.019453113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:21.452553017 +0000 UTC m=+44.695630850" watchObservedRunningTime="2026-06-11 13:11:21.453596841 +0000 UTC m=+44.696674729" Jun 11 13:11:21.521321 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.521295 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:21.559795 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.559768 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-gxp8r"] Jun 11 13:11:21.563019 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:21.562998 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2681592e_0c00_42e9_947d_a313fb719677.slice/crio-531da1b1c9d187d52857f5cb99b9e3768192c010f7912a7e86e7579cf5729b10 WatchSource:0}: Error finding container 531da1b1c9d187d52857f5cb99b9e3768192c010f7912a7e86e7579cf5729b10: Status 404 returned error can't find the container with id 531da1b1c9d187d52857f5cb99b9e3768192c010f7912a7e86e7579cf5729b10 Jun 11 13:11:21.621050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:21.620981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:22.102530 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:22.102492 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:22.443823 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:22.443791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" event={"ID":"2681592e-0c00-42e9-947d-a313fb719677","Type":"ContainerStarted","Data":"531da1b1c9d187d52857f5cb99b9e3768192c010f7912a7e86e7579cf5729b10"} Jun 11 13:11:24.219589 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:24.219551 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:25.454702 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:25.454665 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" event={"ID":"2681592e-0c00-42e9-947d-a313fb719677","Type":"ContainerStarted","Data":"2df1de395e3980953fdacb3945151ba66e192b6ea4ab4a7ed74f80b822feb51b"} Jun 11 13:11:25.471466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:25.471406 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-gxp8r" podStartSLOduration=1.694994188 podStartE2EDuration="4.47138701s" totalImagesPullingTime="2.776392822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:25.470645196 +0000 UTC m=+48.713723049" watchObservedRunningTime="2026-06-11 13:11:25.47138701 +0000 UTC m=+48.714464845" Jun 11 13:11:25.526467 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:25.526430 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:25.676246 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:25.676216 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:25.886198 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:25.886073 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:25.942300 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:25.942272 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:26.031313 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:26.031286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:26.127257 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:26.127223 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:26.227158 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:26.227132 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:27.012302 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:27.012267 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:27.416775 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:27.416711 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:27.813018 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:27.812913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:28.217743 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:28.217710 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:28.432823 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:28.432795 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:28.613705 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:28.613642 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:28.714135 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:28.714096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:28.941045 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:28.941017 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls\") pod \"image-registry-76c675c55d-cqxtm\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:28.941187 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:28.941064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:28.941187 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:28.941152 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 13:11:28.941187 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:28.941157 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 13:11:28.941187 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:28.941174 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c675c55d-cqxtm: secret "image-registry-tls" not found Jun 11 13:11:28.941310 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:28.941200 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls podName:2d7fa167-12b1-4e91-93e6-0722528075d2 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:44.941187603 +0000 UTC m=+68.184265414 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls") pod "dns-default-6bbwn" (UID: "2d7fa167-12b1-4e91-93e6-0722528075d2") : secret "dns-default-metrics-tls" not found Jun 11 13:11:28.941310 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:28.941220 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls podName:602cc0fa-d5b5-49db-bd64-cb8c85b52291 nodeName:}" failed. No retries permitted until 2026-06-11 13:11:44.941205421 +0000 UTC m=+68.184283245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls") pod "image-registry-76c675c55d-cqxtm" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291") : secret "image-registry-tls" not found Jun 11 13:11:29.232729 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:29.232644 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:29.462746 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:29.462712 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:29.811766 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:29.811741 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:30.011813 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:30.011784 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:30.217800 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:30.217773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:30.279320 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:30.279285 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:30.421563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:30.421535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:30.612634 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:30.612556 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:30.713533 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:30.713512 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:30.774791 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:30.774772 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:31.086576 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:31.086548 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:31.685148 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:31.685108 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:31.816866 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:31.816832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:32.012327 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:32.012268 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:32.214209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:32.214183 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:32.416803 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:32.416767 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:32.618128 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:32.618099 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:33.620230 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:33.620198 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.015509 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.015476 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.214561 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.214528 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.315757 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.315697 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.369536 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.369512 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.421932 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.421906 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.523085 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.523063 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.816032 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.816005 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.915958 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.915923 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:34.965293 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:34.965264 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:35.286883 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:35.286848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:35.335696 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:35.335666 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:35.486848 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:35.486819 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:35.615738 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:35.615663 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:35.716954 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:35.716920 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:35.816670 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:35.816640 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:35.917897 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:35.917867 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:36.018536 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:36.018504 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:36.214762 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:36.214685 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:36.454028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:36.453995 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:36.506428 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:36.506370 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:36.612099 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:36.612071 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:37.313223 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:37.313192 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:37.414721 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:37.414695 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:39.326237 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:39.326207 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxgg2" Jun 11 13:11:42.937784 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.937749 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q"] Jun 11 13:11:42.940124 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.940095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:42.940265 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.940146 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:42.940265 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.940179 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:42.943741 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.943720 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 13:11:42.944753 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.944737 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 13:11:42.944814 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.944758 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 13:11:42.953271 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.953250 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 13:11:42.953830 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.953807 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fddd497e-da5a-4f4f-adcf-7041387c2d92-original-pull-secret\") pod \"global-pull-secret-syncer-6xs65\" (UID: \"fddd497e-da5a-4f4f-adcf-7041387c2d92\") " pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:42.953973 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.953952 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9364f758-a646-4d6c-902a-9e361a572682-metrics-certs\") pod \"network-metrics-daemon-lwnxx\" (UID: \"9364f758-a646-4d6c-902a-9e361a572682\") " pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:42.963814 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.963786 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sg2b\" (UniqueName: \"kubernetes.io/projected/34a4f037-4b0b-4ffc-8e07-3e47dee50718-kube-api-access-6sg2b\") pod \"network-check-target-rw5vd\" (UID: \"34a4f037-4b0b-4ffc-8e07-3e47dee50718\") " pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:42.963917 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.963811 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:42.969842 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.969823 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc"] Jun 11 13:11:42.969967 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.969951 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:42.972584 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.972563 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 11 13:11:42.972706 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.972588 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 11 13:11:42.972706 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.972567 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 11 13:11:42.972819 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.972710 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 11 13:11:42.991855 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.991833 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76c675c55d-cqxtm"] Jun 11 13:11:42.991954 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.991873 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q"] Jun 11 13:11:42.991954 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.991890 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc"] Jun 11 13:11:42.992048 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.991963 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" Jun 11 13:11:42.992048 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:42.992020 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" podUID="602cc0fa-d5b5-49db-bd64-cb8c85b52291" Jun 11 13:11:42.994607 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.994587 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 11 13:11:42.994607 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:42.994595 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fnwfp\"" Jun 11 13:11:43.024712 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.024694 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6xs65" Jun 11 13:11:43.031693 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.031669 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-stw6h"] Jun 11 13:11:43.033878 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.033863 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2fc6r\"" Jun 11 13:11:43.037726 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.037709 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lb4sh\"" Jun 11 13:11:43.039748 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.039725 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:43.041296 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.041274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/101725fd-5485-4ebe-b4da-4a554ee67944-tmp\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.041366 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.041317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8tcb\" (UniqueName: \"kubernetes.io/projected/51cc145a-2961-48e7-96fa-491e3bbfbeb3-kube-api-access-x8tcb\") pod \"managed-serviceaccount-addon-agent-848df4646b-jhjxc\" (UID: \"51cc145a-2961-48e7-96fa-491e3bbfbeb3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" Jun 11 13:11:43.041366 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.041341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/101725fd-5485-4ebe-b4da-4a554ee67944-klusterlet-config\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.041366 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.041358 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlw2v\" (UniqueName: \"kubernetes.io/projected/101725fd-5485-4ebe-b4da-4a554ee67944-kube-api-access-xlw2v\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.041464 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.041426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51cc145a-2961-48e7-96fa-491e3bbfbeb3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-848df4646b-jhjxc\" (UID: \"51cc145a-2961-48e7-96fa-491e3bbfbeb3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" Jun 11 13:11:43.042218 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.042195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwnxx" Jun 11 13:11:43.045931 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.045914 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:11:43.055714 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.055684 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bf4d849d4-skq4q"] Jun 11 13:11:43.056006 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.055905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-stw6h" Jun 11 13:11:43.060071 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.060050 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 13:11:43.060183 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.060051 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8wlwn\"" Jun 11 13:11:43.060183 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.060086 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 13:11:43.078206 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.078186 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sjjbn"] Jun 11 13:11:43.078351 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.078335 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.081761 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.081741 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4zbfm\"" Jun 11 13:11:43.081879 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.081777 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 13:11:43.081879 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.081779 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 13:11:43.082002 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.081903 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 13:11:43.082057 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.082002 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 13:11:43.082120 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.082106 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 13:11:43.111050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.110973 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:43.118717 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.118683 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55b4b6869f-frrp6"] Jun 11 13:11:43.119781 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.119042 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.126904 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.123694 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 13:11:43.126904 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.123738 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 13:11:43.126904 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.124388 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 13:11:43.126904 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.124714 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w9bx2\"" Jun 11 13:11:43.126904 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.125489 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 13:11:43.142043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.141908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx6b4\" (UniqueName: \"kubernetes.io/projected/78ee82ec-97dc-4224-b1e8-6dc424124f41-kube-api-access-lx6b4\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.142043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.141964 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ee82ec-97dc-4224-b1e8-6dc424124f41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.142043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.141998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/101725fd-5485-4ebe-b4da-4a554ee67944-klusterlet-config\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.142258 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78ee82ec-97dc-4224-b1e8-6dc424124f41-crio-socket\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142377 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-console-config\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142431 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-oauth-serving-cert\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csf6t\" (UniqueName: \"kubernetes.io/projected/8faa9812-3a93-4db1-a37d-e906a572f632-kube-api-access-csf6t\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142478 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9wvb\" (UniqueName: \"kubernetes.io/projected/032c44d5-58a5-4dd2-8c67-97e4828bfbcb-kube-api-access-r9wvb\") pod \"downloads-6657875f97-stw6h\" (UID: \"032c44d5-58a5-4dd2-8c67-97e4828bfbcb\") " pod="openshift-console/downloads-6657875f97-stw6h" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-service-ca\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/101725fd-5485-4ebe-b4da-4a554ee67944-tmp\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-oauth-config\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8tcb\" (UniqueName: \"kubernetes.io/projected/51cc145a-2961-48e7-96fa-491e3bbfbeb3-kube-api-access-x8tcb\") pod \"managed-serviceaccount-addon-agent-848df4646b-jhjxc\" (UID: \"51cc145a-2961-48e7-96fa-491e3bbfbeb3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142641 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78ee82ec-97dc-4224-b1e8-6dc424124f41-data-volume\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142665 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ee82ec-97dc-4224-b1e8-6dc424124f41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142694 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlw2v\" (UniqueName: \"kubernetes.io/projected/101725fd-5485-4ebe-b4da-4a554ee67944-kube-api-access-xlw2v\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.142792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-serving-cert\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.143471 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.143134 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/101725fd-5485-4ebe-b4da-4a554ee67944-tmp\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.143962 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.142750 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51cc145a-2961-48e7-96fa-491e3bbfbeb3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-848df4646b-jhjxc\" (UID: \"51cc145a-2961-48e7-96fa-491e3bbfbeb3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" Jun 11 13:11:43.146843 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.146801 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51cc145a-2961-48e7-96fa-491e3bbfbeb3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-848df4646b-jhjxc\" (UID: \"51cc145a-2961-48e7-96fa-491e3bbfbeb3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" Jun 11 13:11:43.150567 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.150499 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/101725fd-5485-4ebe-b4da-4a554ee67944-klusterlet-config\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.152487 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.152440 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8tcb\" (UniqueName: \"kubernetes.io/projected/51cc145a-2961-48e7-96fa-491e3bbfbeb3-kube-api-access-x8tcb\") pod \"managed-serviceaccount-addon-agent-848df4646b-jhjxc\" (UID: \"51cc145a-2961-48e7-96fa-491e3bbfbeb3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" Jun 11 13:11:43.152950 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.152849 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-stw6h"] Jun 11 13:11:43.152950 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.152891 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bf4d849d4-skq4q"] Jun 11 13:11:43.152950 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.152915 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sjjbn"] Jun 11 13:11:43.152950 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.152949 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55b4b6869f-frrp6"] Jun 11 13:11:43.153281 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.153063 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.153281 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.153131 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlw2v\" (UniqueName: \"kubernetes.io/projected/101725fd-5485-4ebe-b4da-4a554ee67944-kube-api-access-xlw2v\") pod \"klusterlet-addon-workmgr-795c5c4dd5-j4h5q\" (UID: \"101725fd-5485-4ebe-b4da-4a554ee67944\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.208340 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.208282 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6xs65"] Jun 11 13:11:43.213609 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:43.213579 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfddd497e_da5a_4f4f_adcf_7041387c2d92.slice/crio-5d783e93823588dd7c60ece026e8334a1e709346168cfc1738de2a569bd1bb3d WatchSource:0}: Error finding container 5d783e93823588dd7c60ece026e8334a1e709346168cfc1738de2a569bd1bb3d: Status 404 returned error can't find the container with id 5d783e93823588dd7c60ece026e8334a1e709346168cfc1738de2a569bd1bb3d Jun 11 13:11:43.219278 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.219187 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:43.231521 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.231496 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rw5vd"] Jun 11 13:11:43.234801 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:43.234780 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a4f037_4b0b_4ffc_8e07_3e47dee50718.slice/crio-2e7512cf89802ca0e09dfc5717707de20f27e23687afb4d72f1d759f0457b495 WatchSource:0}: Error finding container 2e7512cf89802ca0e09dfc5717707de20f27e23687afb4d72f1d759f0457b495: Status 404 returned error can't find the container with id 2e7512cf89802ca0e09dfc5717707de20f27e23687afb4d72f1d759f0457b495 Jun 11 13:11:43.243818 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.243800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-serving-cert\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.243905 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.243832 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d3b8ca2e-bc93-4231-8915-80e0457a23c6-image-registry-private-configuration\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.243905 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.243853 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-bound-sa-token\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.243905 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.243873 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx6b4\" (UniqueName: \"kubernetes.io/projected/78ee82ec-97dc-4224-b1e8-6dc424124f41-kube-api-access-lx6b4\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.243905 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.243892 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d3b8ca2e-bc93-4231-8915-80e0457a23c6-installation-pull-secrets\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.244097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.243933 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ee82ec-97dc-4224-b1e8-6dc424124f41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.244097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.243963 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78ee82ec-97dc-4224-b1e8-6dc424124f41-crio-socket\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.244097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244002 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-console-config\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.244097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-registry-tls\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.244097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244030 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78ee82ec-97dc-4224-b1e8-6dc424124f41-crio-socket\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.244097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244057 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3b8ca2e-bc93-4231-8915-80e0457a23c6-trusted-ca\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.244398 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-oauth-serving-cert\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.244398 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244135 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csf6t\" (UniqueName: \"kubernetes.io/projected/8faa9812-3a93-4db1-a37d-e906a572f632-kube-api-access-csf6t\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.244398 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244165 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d3b8ca2e-bc93-4231-8915-80e0457a23c6-registry-certificates\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.244398 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244192 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9wvb\" (UniqueName: \"kubernetes.io/projected/032c44d5-58a5-4dd2-8c67-97e4828bfbcb-kube-api-access-r9wvb\") pod \"downloads-6657875f97-stw6h\" (UID: \"032c44d5-58a5-4dd2-8c67-97e4828bfbcb\") " pod="openshift-console/downloads-6657875f97-stw6h" Jun 11 13:11:43.244398 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244217 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-service-ca\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.244398 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-oauth-config\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.244398 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244314 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d3b8ca2e-bc93-4231-8915-80e0457a23c6-ca-trust-extracted\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.244398 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244342 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7szm8\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-kube-api-access-7szm8\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.245058 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244508 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78ee82ec-97dc-4224-b1e8-6dc424124f41-data-volume\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.245058 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244539 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ee82ec-97dc-4224-b1e8-6dc424124f41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.245058 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244787 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-console-config\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.245058 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.244848 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-oauth-serving-cert\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.245238 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.245058 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-service-ca\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.245238 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.245137 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ee82ec-97dc-4224-b1e8-6dc424124f41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.245238 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.245179 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78ee82ec-97dc-4224-b1e8-6dc424124f41-data-volume\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.246361 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.246341 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ee82ec-97dc-4224-b1e8-6dc424124f41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.246860 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.246836 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-serving-cert\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.246939 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.246889 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-oauth-config\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.253411 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.253389 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx6b4\" (UniqueName: \"kubernetes.io/projected/78ee82ec-97dc-4224-b1e8-6dc424124f41-kube-api-access-lx6b4\") pod \"insights-runtime-extractor-sjjbn\" (UID: \"78ee82ec-97dc-4224-b1e8-6dc424124f41\") " pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.254056 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.254034 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9wvb\" (UniqueName: \"kubernetes.io/projected/032c44d5-58a5-4dd2-8c67-97e4828bfbcb-kube-api-access-r9wvb\") pod \"downloads-6657875f97-stw6h\" (UID: \"032c44d5-58a5-4dd2-8c67-97e4828bfbcb\") " pod="openshift-console/downloads-6657875f97-stw6h" Jun 11 13:11:43.254216 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.254200 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csf6t\" (UniqueName: \"kubernetes.io/projected/8faa9812-3a93-4db1-a37d-e906a572f632-kube-api-access-csf6t\") pod \"console-bf4d849d4-skq4q\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.259232 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.259214 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lwnxx"] Jun 11 13:11:43.261602 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:43.261575 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9364f758_a646_4d6c_902a_9e361a572682.slice/crio-24cda9663058d02bd4545553a79e835167ba7550970ba031d6b15ff1fe12aebf WatchSource:0}: Error finding container 24cda9663058d02bd4545553a79e835167ba7550970ba031d6b15ff1fe12aebf: Status 404 returned error can't find the container with id 24cda9663058d02bd4545553a79e835167ba7550970ba031d6b15ff1fe12aebf Jun 11 13:11:43.279043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.279012 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:11:43.309844 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.309820 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" Jun 11 13:11:43.318414 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.318386 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:43.345465 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.345440 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-registry-tls\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.345523 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.345482 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3b8ca2e-bc93-4231-8915-80e0457a23c6-trusted-ca\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.345562 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.345533 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d3b8ca2e-bc93-4231-8915-80e0457a23c6-registry-certificates\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.345631 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.345563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d3b8ca2e-bc93-4231-8915-80e0457a23c6-ca-trust-extracted\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.345631 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.345589 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7szm8\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-kube-api-access-7szm8\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.345699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.345677 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d3b8ca2e-bc93-4231-8915-80e0457a23c6-image-registry-private-configuration\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.345735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.345709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-bound-sa-token\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.345771 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.345743 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d3b8ca2e-bc93-4231-8915-80e0457a23c6-installation-pull-secrets\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.346450 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.346392 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d3b8ca2e-bc93-4231-8915-80e0457a23c6-ca-trust-extracted\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.346667 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.346641 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d3b8ca2e-bc93-4231-8915-80e0457a23c6-registry-certificates\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.347140 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.347115 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3b8ca2e-bc93-4231-8915-80e0457a23c6-trusted-ca\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.348264 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.348244 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-registry-tls\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.348535 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.348515 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d3b8ca2e-bc93-4231-8915-80e0457a23c6-image-registry-private-configuration\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.348764 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.348742 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d3b8ca2e-bc93-4231-8915-80e0457a23c6-installation-pull-secrets\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.353497 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.353477 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-bound-sa-token\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.353987 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.353968 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7szm8\" (UniqueName: \"kubernetes.io/projected/d3b8ca2e-bc93-4231-8915-80e0457a23c6-kube-api-access-7szm8\") pod \"image-registry-55b4b6869f-frrp6\" (UID: \"d3b8ca2e-bc93-4231-8915-80e0457a23c6\") " pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.371781 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.371738 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-stw6h" Jun 11 13:11:43.387846 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.387820 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:11:43.416765 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.416737 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:43.435115 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.434906 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sjjbn" Jun 11 13:11:43.445056 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.444819 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q"] Jun 11 13:11:43.448514 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:43.448471 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod101725fd_5485_4ebe_b4da_4a554ee67944.slice/crio-305c012c816a8502657cb45e50cb26223b2a82dd3940bb969df6aed3e6eafe93 WatchSource:0}: Error finding container 305c012c816a8502657cb45e50cb26223b2a82dd3940bb969df6aed3e6eafe93: Status 404 returned error can't find the container with id 305c012c816a8502657cb45e50cb26223b2a82dd3940bb969df6aed3e6eafe93 Jun 11 13:11:43.471220 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.471184 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9w7l7\"" Jun 11 13:11:43.477773 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.477749 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:43.488333 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.488275 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc"] Jun 11 13:11:43.502279 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.502251 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" event={"ID":"101725fd-5485-4ebe-b4da-4a554ee67944","Type":"ContainerStarted","Data":"305c012c816a8502657cb45e50cb26223b2a82dd3940bb969df6aed3e6eafe93"} Jun 11 13:11:43.504093 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.503778 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rw5vd" event={"ID":"34a4f037-4b0b-4ffc-8e07-3e47dee50718","Type":"ContainerStarted","Data":"2e7512cf89802ca0e09dfc5717707de20f27e23687afb4d72f1d759f0457b495"} Jun 11 13:11:43.505209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.505143 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6xs65" event={"ID":"fddd497e-da5a-4f4f-adcf-7041387c2d92","Type":"ContainerStarted","Data":"5d783e93823588dd7c60ece026e8334a1e709346168cfc1738de2a569bd1bb3d"} Jun 11 13:11:43.506287 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.506264 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" event={"ID":"51cc145a-2961-48e7-96fa-491e3bbfbeb3","Type":"ContainerStarted","Data":"95a764ca849474ea961bd9c07759b29cebff7f02b00a2e8f05360fd3c0b53488"} Jun 11 13:11:43.508436 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.507705 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwnxx" event={"ID":"9364f758-a646-4d6c-902a-9e361a572682","Type":"ContainerStarted","Data":"24cda9663058d02bd4545553a79e835167ba7550970ba031d6b15ff1fe12aebf"} Jun 11 13:11:43.508436 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.507746 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:43.539457 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.539195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:43.552093 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.552066 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-stw6h"] Jun 11 13:11:43.555138 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:43.555106 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod032c44d5_58a5_4dd2_8c67_97e4828bfbcb.slice/crio-c5b82a5a9d0af4560725459f094e519334e0fbc6fd503cd276b21a8776a35e26 WatchSource:0}: Error finding container c5b82a5a9d0af4560725459f094e519334e0fbc6fd503cd276b21a8776a35e26: Status 404 returned error can't find the container with id c5b82a5a9d0af4560725459f094e519334e0fbc6fd503cd276b21a8776a35e26 Jun 11 13:11:43.580777 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.580697 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bf4d849d4-skq4q"] Jun 11 13:11:43.589131 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:43.589107 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8faa9812_3a93_4db1_a37d_e906a572f632.slice/crio-3848c8de34e0530f75253059a1407a098ef6c4534c60717041b6a6a82d5ca0b6 WatchSource:0}: Error finding container 3848c8de34e0530f75253059a1407a098ef6c4534c60717041b6a6a82d5ca0b6: Status 404 returned error can't find the container with id 3848c8de34e0530f75253059a1407a098ef6c4534c60717041b6a6a82d5ca0b6 Jun 11 13:11:43.638159 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.638127 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sjjbn"] Jun 11 13:11:43.641950 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:43.641919 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ee82ec_97dc_4224_b1e8_6dc424124f41.slice/crio-72614f389316b348feb24a4b442eb846a68bd719f2620dbc7f8b424692939de8 WatchSource:0}: Error finding container 72614f389316b348feb24a4b442eb846a68bd719f2620dbc7f8b424692939de8: Status 404 returned error can't find the container with id 72614f389316b348feb24a4b442eb846a68bd719f2620dbc7f8b424692939de8 Jun 11 13:11:43.650563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.650538 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4dh6\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-kube-api-access-g4dh6\") pod \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " Jun 11 13:11:43.650705 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.650657 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-bound-sa-token\") pod \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " Jun 11 13:11:43.650770 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.650707 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-image-registry-private-configuration\") pod \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " Jun 11 13:11:43.650770 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.650746 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-certificates\") pod \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " Jun 11 13:11:43.650770 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.650764 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-trusted-ca\") pod \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " Jun 11 13:11:43.650975 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.650782 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-installation-pull-secrets\") pod \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " Jun 11 13:11:43.650975 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.650802 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/602cc0fa-d5b5-49db-bd64-cb8c85b52291-ca-trust-extracted\") pod \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\" (UID: \"602cc0fa-d5b5-49db-bd64-cb8c85b52291\") " Jun 11 13:11:43.651340 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.651313 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "602cc0fa-d5b5-49db-bd64-cb8c85b52291" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:11:43.651403 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.651354 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602cc0fa-d5b5-49db-bd64-cb8c85b52291-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "602cc0fa-d5b5-49db-bd64-cb8c85b52291" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 13:11:43.652264 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.652235 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "602cc0fa-d5b5-49db-bd64-cb8c85b52291" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:11:43.653157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.653111 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-kube-api-access-g4dh6" (OuterVolumeSpecName: "kube-api-access-g4dh6") pod "602cc0fa-d5b5-49db-bd64-cb8c85b52291" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291"). InnerVolumeSpecName "kube-api-access-g4dh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:11:43.653435 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.653410 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "602cc0fa-d5b5-49db-bd64-cb8c85b52291" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:11:43.654177 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.654151 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "602cc0fa-d5b5-49db-bd64-cb8c85b52291" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:11:43.654397 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.654370 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "602cc0fa-d5b5-49db-bd64-cb8c85b52291" (UID: "602cc0fa-d5b5-49db-bd64-cb8c85b52291"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:11:43.672241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.672217 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55b4b6869f-frrp6"] Jun 11 13:11:43.676891 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:43.676864 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b8ca2e_bc93_4231_8915_80e0457a23c6.slice/crio-9b831223579390613eb3fcb40d43b88d8d0bccd4af11edb8586e76c53844784c WatchSource:0}: Error finding container 9b831223579390613eb3fcb40d43b88d8d0bccd4af11edb8586e76c53844784c: Status 404 returned error can't find the container with id 9b831223579390613eb3fcb40d43b88d8d0bccd4af11edb8586e76c53844784c Jun 11 13:11:43.752401 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.752275 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-certificates\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:11:43.752401 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.752302 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/602cc0fa-d5b5-49db-bd64-cb8c85b52291-trusted-ca\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:11:43.752401 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.752319 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-installation-pull-secrets\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:11:43.752664 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.752507 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/602cc0fa-d5b5-49db-bd64-cb8c85b52291-ca-trust-extracted\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:11:43.752664 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.752526 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4dh6\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-kube-api-access-g4dh6\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:11:43.752664 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.752540 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-bound-sa-token\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:11:43.752664 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:43.752554 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/602cc0fa-d5b5-49db-bd64-cb8c85b52291-image-registry-private-configuration\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:11:44.519548 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.518934 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" event={"ID":"d3b8ca2e-bc93-4231-8915-80e0457a23c6","Type":"ContainerStarted","Data":"4d154a05d3b18bce7b7fca8d64daf39874fbbc968b20c2da507ae45c2f777478"} Jun 11 13:11:44.519548 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.518994 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" event={"ID":"d3b8ca2e-bc93-4231-8915-80e0457a23c6","Type":"ContainerStarted","Data":"9b831223579390613eb3fcb40d43b88d8d0bccd4af11edb8586e76c53844784c"} Jun 11 13:11:44.519548 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.519417 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:11:44.528214 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.528176 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-stw6h" event={"ID":"032c44d5-58a5-4dd2-8c67-97e4828bfbcb","Type":"ContainerStarted","Data":"c5b82a5a9d0af4560725459f094e519334e0fbc6fd503cd276b21a8776a35e26"} Jun 11 13:11:44.531298 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.531272 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjjbn" event={"ID":"78ee82ec-97dc-4224-b1e8-6dc424124f41","Type":"ContainerStarted","Data":"1c21f54ba594a2ba8d6dccf7b5eb722c02601deddfd5888558c3010c8a78a0ee"} Jun 11 13:11:44.531422 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.531341 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjjbn" event={"ID":"78ee82ec-97dc-4224-b1e8-6dc424124f41","Type":"ContainerStarted","Data":"72614f389316b348feb24a4b442eb846a68bd719f2620dbc7f8b424692939de8"} Jun 11 13:11:44.534077 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.534052 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76c675c55d-cqxtm" Jun 11 13:11:44.534185 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.534076 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf4d849d4-skq4q" event={"ID":"8faa9812-3a93-4db1-a37d-e906a572f632","Type":"ContainerStarted","Data":"3848c8de34e0530f75253059a1407a098ef6c4534c60717041b6a6a82d5ca0b6"} Jun 11 13:11:44.544736 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.544692 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" podStartSLOduration=1.5446750599999999 podStartE2EDuration="1.54467506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:44.543125839 +0000 UTC m=+67.786203679" watchObservedRunningTime="2026-06-11 13:11:44.54467506 +0000 UTC m=+67.787752897" Jun 11 13:11:44.574336 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.574308 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76c675c55d-cqxtm"] Jun 11 13:11:44.579835 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.579809 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76c675c55d-cqxtm"] Jun 11 13:11:44.683897 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.683853 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/602cc0fa-d5b5-49db-bd64-cb8c85b52291-registry-tls\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:11:44.987419 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.986979 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:44.996795 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:44.996760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d7fa167-12b1-4e91-93e6-0722528075d2-metrics-tls\") pod \"dns-default-6bbwn\" (UID: \"2d7fa167-12b1-4e91-93e6-0722528075d2\") " pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:45.216368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.216165 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s9zw4\"" Jun 11 13:11:45.223590 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.223556 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="602cc0fa-d5b5-49db-bd64-cb8c85b52291" path="/var/lib/kubelet/pods/602cc0fa-d5b5-49db-bd64-cb8c85b52291/volumes" Jun 11 13:11:45.223864 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.223847 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6bbwn" Jun 11 13:11:45.323147 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.323068 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-687799c7f-lq679"] Jun 11 13:11:45.326101 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.326079 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.337782 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.337549 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-687799c7f-lq679"] Jun 11 13:11:45.337965 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.337941 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 11 13:11:45.341411 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.340830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:45.391157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.390320 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-oauth-serving-cert\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.391157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.390365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-oauth-config\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.391157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.390393 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-service-ca\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.391157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.390430 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-trusted-ca-bundle\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.391157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.390454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-serving-cert\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.391157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.390485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7wdk\" (UniqueName: \"kubernetes.io/projected/6b9d2994-71b3-4437-b9ea-578f0e19acc8-kube-api-access-l7wdk\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.391157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.390523 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-config\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.391157 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.391068 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.491924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-serving-cert\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.491988 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7wdk\" (UniqueName: \"kubernetes.io/projected/6b9d2994-71b3-4437-b9ea-578f0e19acc8-kube-api-access-l7wdk\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.492036 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-config\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.492094 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-oauth-serving-cert\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.492125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-oauth-config\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.492150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-service-ca\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.492193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-trusted-ca-bundle\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.493194 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-trusted-ca-bundle\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.493289 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-config\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.493796 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-oauth-serving-cert\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.494348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.494309 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-service-ca\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.509886 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.509833 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-serving-cert\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.514698 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.514654 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-oauth-config\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.516196 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.516146 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7wdk\" (UniqueName: \"kubernetes.io/projected/6b9d2994-71b3-4437-b9ea-578f0e19acc8-kube-api-access-l7wdk\") pod \"console-687799c7f-lq679\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:45.546040 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.545604 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjjbn" event={"ID":"78ee82ec-97dc-4224-b1e8-6dc424124f41","Type":"ContainerStarted","Data":"39eb8a7f5ff6da71a811aca0e37a94d3df28ba6b1b157f521c95f17d190285f2"} Jun 11 13:11:45.645089 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:45.644562 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687799c7f-lq679" Jun 11 13:11:46.317244 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.317164 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6bbwn"] Jun 11 13:11:46.332227 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:46.332196 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d7fa167_12b1_4e91_93e6_0722528075d2.slice/crio-ec47c9a0972cedb7c32a3c946def82d5954c46f0974f03c6178e97494c4bbdda WatchSource:0}: Error finding container ec47c9a0972cedb7c32a3c946def82d5954c46f0974f03c6178e97494c4bbdda: Status 404 returned error can't find the container with id ec47c9a0972cedb7c32a3c946def82d5954c46f0974f03c6178e97494c4bbdda Jun 11 13:11:46.362554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.362529 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-687799c7f-lq679"] Jun 11 13:11:46.368834 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:11:46.368797 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b9d2994_71b3_4437_b9ea_578f0e19acc8.slice/crio-57e8c59266a6ec4ea84f000e3553b8b04d5de31085a6a2433088704229adf1d1 WatchSource:0}: Error finding container 57e8c59266a6ec4ea84f000e3553b8b04d5de31085a6a2433088704229adf1d1: Status 404 returned error can't find the container with id 57e8c59266a6ec4ea84f000e3553b8b04d5de31085a6a2433088704229adf1d1 Jun 11 13:11:46.568494 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.568402 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwnxx" event={"ID":"9364f758-a646-4d6c-902a-9e361a572682","Type":"ContainerStarted","Data":"d880aed3f5cda0ebb6b4b27d73c36eec7097eaae66d25ee24a8f02a13312c67d"} Jun 11 13:11:46.575444 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.575411 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6bbwn" event={"ID":"2d7fa167-12b1-4e91-93e6-0722528075d2","Type":"ContainerStarted","Data":"ec47c9a0972cedb7c32a3c946def82d5954c46f0974f03c6178e97494c4bbdda"} Jun 11 13:11:46.577720 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.577672 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687799c7f-lq679" event={"ID":"6b9d2994-71b3-4437-b9ea-578f0e19acc8","Type":"ContainerStarted","Data":"57e8c59266a6ec4ea84f000e3553b8b04d5de31085a6a2433088704229adf1d1"} Jun 11 13:11:46.621207 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.621158 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:46.784481 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.784391 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:46.795726 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.794113 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr"] Jun 11 13:11:46.797936 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.797192 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" Jun 11 13:11:46.801142 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.801117 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gdp6t\"" Jun 11 13:11:46.801365 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.801349 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 11 13:11:46.805236 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.805214 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr"] Jun 11 13:11:46.856126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.856062 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:46.914699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:46.914591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/47eafdac-30ee-4c3c-9776-20e525434f7c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-5zhvr\" (UID: \"47eafdac-30ee-4c3c-9776-20e525434f7c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" Jun 11 13:11:47.016373 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.015792 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/47eafdac-30ee-4c3c-9776-20e525434f7c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-5zhvr\" (UID: \"47eafdac-30ee-4c3c-9776-20e525434f7c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" Jun 11 13:11:47.016373 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:47.016040 2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 11 13:11:47.016373 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:11:47.016118 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47eafdac-30ee-4c3c-9776-20e525434f7c-tls-certificates podName:47eafdac-30ee-4c3c-9776-20e525434f7c nodeName:}" failed. No retries permitted until 2026-06-11 13:11:47.516090842 +0000 UTC m=+70.759168658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/47eafdac-30ee-4c3c-9776-20e525434f7c-tls-certificates") pod "prometheus-operator-admission-webhook-5b6b8f594-5zhvr" (UID: "47eafdac-30ee-4c3c-9776-20e525434f7c") : secret "prometheus-operator-admission-webhook-tls" not found Jun 11 13:11:47.283389 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.283333 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:47.520209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.520165 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/47eafdac-30ee-4c3c-9776-20e525434f7c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-5zhvr\" (UID: \"47eafdac-30ee-4c3c-9776-20e525434f7c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" Jun 11 13:11:47.534456 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.534370 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/47eafdac-30ee-4c3c-9776-20e525434f7c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-5zhvr\" (UID: \"47eafdac-30ee-4c3c-9776-20e525434f7c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" Jun 11 13:11:47.591647 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.591254 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwnxx" event={"ID":"9364f758-a646-4d6c-902a-9e361a572682","Type":"ContainerStarted","Data":"ca8c3d3d37d7be40344b6661ea4dc3a3af160bd14da1bf1e989972241cb33648"} Jun 11 13:11:47.609144 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.608590 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lwnxx" podStartSLOduration=67.75945484 podStartE2EDuration="1m10.608560462s" totalImagesPullingTime="2.849105622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:11:47.606671551 +0000 UTC m=+70.849749383" watchObservedRunningTime="2026-06-11 13:11:47.608560462 +0000 UTC m=+70.851638298" Jun 11 13:11:47.693766 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.693711 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:47.719857 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.719820 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" Jun 11 13:11:47.747436 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.747379 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:47.797833 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:47.797536 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:49.781642 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:49.781584 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:49.884541 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:49.884492 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:49.988206 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:49.988172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:50.083657 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:50.083556 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:50.146969 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:50.146937 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:50.213850 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:50.213793 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:50.315127 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:50.315097 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:50.996221 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:50.996180 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:51.050768 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:51.050730 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:51.394954 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:51.394876 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:51.496201 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:51.496167 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:54.351972 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:54.351937 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:54.408976 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:54.408813 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:11:56.870133 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:56.870071 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr"] Jun 11 13:11:57.328940 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:11:57.328904 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:03.482478 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:03.482420 2563 patch_prober.go:28] interesting pod/image-registry-55b4b6869f-frrp6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 13:12:03.482984 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:03.482502 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" podUID="d3b8ca2e-bc93-4231-8915-80e0457a23c6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 13:12:03.798516 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:03.798428 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:03.872793 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:03.872765 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:03.949937 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:03.949909 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:04.007741 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.007721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:04.159355 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:04.159324 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47eafdac_30ee_4c3c_9776_20e525434f7c.slice/crio-691e3a9c3fde8bd99789e00666fe17a1299f8f48b2ae9f897ff4b5b780fb83a3 WatchSource:0}: Error finding container 691e3a9c3fde8bd99789e00666fe17a1299f8f48b2ae9f897ff4b5b780fb83a3: Status 404 returned error can't find the container with id 691e3a9c3fde8bd99789e00666fe17a1299f8f48b2ae9f897ff4b5b780fb83a3 Jun 11 13:12:04.673202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.673095 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjjbn" event={"ID":"78ee82ec-97dc-4224-b1e8-6dc424124f41","Type":"ContainerStarted","Data":"b0859f2ec8ce3435a581ec84ae7f1a0de6256988f190062c852f5d293bd685fa"} Jun 11 13:12:04.675151 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.675076 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6bbwn" event={"ID":"2d7fa167-12b1-4e91-93e6-0722528075d2","Type":"ContainerStarted","Data":"1574f7f74d2b0634080e11daa4119f53e9d73245b7959d05cafe7e5a14705cac"} Jun 11 13:12:04.684949 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.684827 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf4d849d4-skq4q" event={"ID":"8faa9812-3a93-4db1-a37d-e906a572f632","Type":"ContainerStarted","Data":"2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116"} Jun 11 13:12:04.686943 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.686882 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" event={"ID":"47eafdac-30ee-4c3c-9776-20e525434f7c","Type":"ContainerStarted","Data":"691e3a9c3fde8bd99789e00666fe17a1299f8f48b2ae9f897ff4b5b780fb83a3"} Jun 11 13:12:04.689808 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.689727 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" event={"ID":"51cc145a-2961-48e7-96fa-491e3bbfbeb3","Type":"ContainerStarted","Data":"55a6a256a49f6e4b53b9e72211038f054b742046351d5250a2a7ff549b0f0239"} Jun 11 13:12:04.697881 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.697421 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sjjbn" podStartSLOduration=1.617094553 podStartE2EDuration="21.697407908s" totalImagesPullingTime="20.080313355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:11:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:04.696910059 +0000 UTC m=+87.939987914" watchObservedRunningTime="2026-06-11 13:12:04.697407908 +0000 UTC m=+87.940485742" Jun 11 13:12:04.739022 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.738528 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bf4d849d4-skq4q" podStartSLOduration=0.913805787 podStartE2EDuration="21.738507382s" totalImagesPullingTime="20.824701595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:04.723315581 +0000 UTC m=+87.966393425" watchObservedRunningTime="2026-06-11 13:12:04.738507382 +0000 UTC m=+87.981585220" Jun 11 13:12:04.740356 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:04.739866 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848df4646b-jhjxc" podStartSLOduration=9.497951759 podStartE2EDuration="22.73985443s" totalImagesPullingTime="13.241902671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:04.739425854 +0000 UTC m=+87.982503683" watchObservedRunningTime="2026-06-11 13:12:04.73985443 +0000 UTC m=+87.982932262" Jun 11 13:12:05.551579 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.551542 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55b4b6869f-frrp6" Jun 11 13:12:05.683875 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.683840 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:05.696537 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.696503 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6bbwn" event={"ID":"2d7fa167-12b1-4e91-93e6-0722528075d2","Type":"ContainerStarted","Data":"85e853cf5bf2c55fec6d738c5f6f6f6381a74406dbbe647b82e5e55d94d443fc"} Jun 11 13:12:05.696724 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.696655 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6bbwn" Jun 11 13:12:05.698417 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.698387 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" event={"ID":"101725fd-5485-4ebe-b4da-4a554ee67944","Type":"ContainerStarted","Data":"ef978dc48f083259c76fd90ebf9863751dba857b8a620d40960c76d52ffc870e"} Jun 11 13:12:05.698576 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.698557 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:12:05.700590 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.700560 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rw5vd" event={"ID":"34a4f037-4b0b-4ffc-8e07-3e47dee50718","Type":"ContainerStarted","Data":"f6965ffc8237eb4b5cc1c4e99c0a58a0db4e3dbf9aedcf24a95524bcdba7b5c5"} Jun 11 13:12:05.700719 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.700683 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" Jun 11 13:12:05.700773 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.700759 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:12:05.702116 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.702092 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6xs65" event={"ID":"fddd497e-da5a-4f4f-adcf-7041387c2d92","Type":"ContainerStarted","Data":"6c2a747b8db7222fea7738b057ad38f43a7c16cfecf930176f99df0724334f73"} Jun 11 13:12:05.703964 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.703936 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687799c7f-lq679" event={"ID":"6b9d2994-71b3-4437-b9ea-578f0e19acc8","Type":"ContainerStarted","Data":"cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299"} Jun 11 13:12:05.705746 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.705722 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-stw6h" event={"ID":"032c44d5-58a5-4dd2-8c67-97e4828bfbcb","Type":"ContainerStarted","Data":"3a8d38f6119b0a77966d031a34ce5d59a234a537df577a696d048469f09a83ae"} Jun 11 13:12:05.714316 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.714266 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6bbwn" podStartSLOduration=35.1699998 podStartE2EDuration="52.714253955s" totalImagesPullingTime="17.544254155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:05.712900391 +0000 UTC m=+88.955978269" watchObservedRunningTime="2026-06-11 13:12:05.714253955 +0000 UTC m=+88.957331813" Jun 11 13:12:05.730888 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.730702 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-687799c7f-lq679" podStartSLOduration=2.69509259 podStartE2EDuration="20.730685279s" totalImagesPullingTime="18.035592689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:05.730282342 +0000 UTC m=+88.973360199" watchObservedRunningTime="2026-06-11 13:12:05.730685279 +0000 UTC m=+88.973763115" Jun 11 13:12:05.746210 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.746161 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795c5c4dd5-j4h5q" podStartSLOduration=2.778415193 podStartE2EDuration="23.746142907s" totalImagesPullingTime="20.967727714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:05.744951952 +0000 UTC m=+88.988029786" watchObservedRunningTime="2026-06-11 13:12:05.746142907 +0000 UTC m=+88.989220741" Jun 11 13:12:05.749947 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.749925 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:05.763122 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.763078 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-stw6h" podStartSLOduration=1.8564479820000002 podStartE2EDuration="22.763063584s" totalImagesPullingTime="20.906615602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:05.761964917 +0000 UTC m=+89.005042770" watchObservedRunningTime="2026-06-11 13:12:05.763063584 +0000 UTC m=+89.006141416" Jun 11 13:12:05.778196 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.778148 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rw5vd" podStartSLOduration=67.597545537 podStartE2EDuration="1m28.778134145s" totalImagesPullingTime="21.180588608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:05.777928269 +0000 UTC m=+89.021006127" watchObservedRunningTime="2026-06-11 13:12:05.778134145 +0000 UTC m=+89.021211981" Jun 11 13:12:05.794305 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.794261 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6xs65" podStartSLOduration=67.592357212 podStartE2EDuration="1m28.794249657s" totalImagesPullingTime="21.201892445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:05.793327551 +0000 UTC m=+89.036405386" watchObservedRunningTime="2026-06-11 13:12:05.794249657 +0000 UTC m=+89.037327490" Jun 11 13:12:05.814887 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.814815 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:05.915856 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.915819 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:05.984331 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:05.984296 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:06.712210 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.712142 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" event={"ID":"47eafdac-30ee-4c3c-9776-20e525434f7c","Type":"ContainerStarted","Data":"cff6787460cffc6b17520d82330c70598104e1245c1563920395fb1f0ca91124"} Jun 11 13:12:06.712663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.712436 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" Jun 11 13:12:06.712885 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.712857 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-stw6h" Jun 11 13:12:06.718310 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.718283 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" Jun 11 13:12:06.720096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.720071 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-stw6h" Jun 11 13:12:06.728136 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.728090 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-5zhvr" podStartSLOduration=18.89627313 podStartE2EDuration="20.728072396s" totalImagesPullingTime="1.831799266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:11:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:06.72789146 +0000 UTC m=+89.970969319" watchObservedRunningTime="2026-06-11 13:12:06.728072396 +0000 UTC m=+89.971150228" Jun 11 13:12:06.818937 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.818901 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:06.863251 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.863216 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx"] Jun 11 13:12:06.881508 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.881477 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx"] Jun 11 13:12:06.881679 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.881614 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:06.884300 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.884281 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 13:12:06.884871 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.884850 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 13:12:06.885003 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.884982 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-97nns\"" Jun 11 13:12:06.885003 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.884870 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 11 13:12:06.885126 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.884945 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 11 13:12:06.885213 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.884982 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 13:12:06.886014 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.885993 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:06.995672 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.995559 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:06.995672 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.995638 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45p84\" (UniqueName: \"kubernetes.io/projected/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-kube-api-access-45p84\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:06.995873 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.995679 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:06.995873 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:06.995771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.096869 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.096827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.097065 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.096918 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.097065 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.096961 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45p84\" (UniqueName: \"kubernetes.io/projected/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-kube-api-access-45p84\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.097065 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.096982 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.097367 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:12:07.097340 2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 11 13:12:07.097441 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:12:07.097419 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-tls podName:5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5 nodeName:}" failed. No retries permitted until 2026-06-11 13:12:07.597402937 +0000 UTC m=+90.840480756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-6jxfx" (UID: "5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5") : secret "prometheus-operator-tls" not found Jun 11 13:12:07.097872 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.097847 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.107060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.107022 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.110501 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.110469 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45p84\" (UniqueName: \"kubernetes.io/projected/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-kube-api-access-45p84\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.387056 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.386971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:07.601527 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.601488 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.604434 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.604403 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-6jxfx\" (UID: \"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.798886 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.798856 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" Jun 11 13:12:07.971385 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:07.971339 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx"] Jun 11 13:12:07.975069 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:07.975038 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfe60de_e57a_4ea4_b4d4_bcc3c8ae46d5.slice/crio-4ecb6cb85d130c7abbb48f371bd0d13a59edadea90353899d1080be22bd0277c WatchSource:0}: Error finding container 4ecb6cb85d130c7abbb48f371bd0d13a59edadea90353899d1080be22bd0277c: Status 404 returned error can't find the container with id 4ecb6cb85d130c7abbb48f371bd0d13a59edadea90353899d1080be22bd0277c Jun 11 13:12:08.723174 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:08.722744 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" event={"ID":"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5","Type":"ContainerStarted","Data":"4ecb6cb85d130c7abbb48f371bd0d13a59edadea90353899d1080be22bd0277c"} Jun 11 13:12:09.725467 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:09.725431 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:09.826585 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:09.826557 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:10.083956 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:10.083873 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:10.733894 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:10.733854 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" event={"ID":"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5","Type":"ContainerStarted","Data":"4c86624bec66d713c853114b6b128b3407e40494ef3e7d94e0eaea856fc5da56"} Jun 11 13:12:10.734443 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:10.733953 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" event={"ID":"5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5","Type":"ContainerStarted","Data":"6d1796fc039d4703a15486754cd95ff5b988651979547259acf91a60a3410a67"} Jun 11 13:12:10.753320 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:10.753267 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-6jxfx" podStartSLOduration=2.9819532090000003 podStartE2EDuration="4.753252948s" totalImagesPullingTime="1.771299739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:10.751086569 +0000 UTC m=+93.994164431" watchObservedRunningTime="2026-06-11 13:12:10.753252948 +0000 UTC m=+93.996330781" Jun 11 13:12:11.967897 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:11.967858 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.187070 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.187033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.201004 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.200978 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb"] Jun 11 13:12:12.229059 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.228988 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb"] Jun 11 13:12:12.229412 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.229397 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l6q47"] Jun 11 13:12:12.230731 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.229326 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.235285 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.234753 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xq4tq\"" Jun 11 13:12:12.235285 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.234960 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 11 13:12:12.235285 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.235136 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 11 13:12:12.237418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.237060 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/de760bf0-c4f8-4281-980c-3abca5f7d0a7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.237418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.237106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de760bf0-c4f8-4281-980c-3abca5f7d0a7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.237418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.237133 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de760bf0-c4f8-4281-980c-3abca5f7d0a7-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.237418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.237186 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vvks\" (UniqueName: \"kubernetes.io/projected/de760bf0-c4f8-4281-980c-3abca5f7d0a7-kube-api-access-8vvks\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.250989 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.250970 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.254571 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.253661 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 13:12:12.254571 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.253897 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-97grj\"" Jun 11 13:12:12.254571 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.254260 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 13:12:12.254571 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.254353 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 13:12:12.281659 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.280148 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.337595 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.337571 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-root\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.338388 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-tls\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.338531 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338517 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-sys\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.338655 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338641 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.338778 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5jbz\" (UniqueName: \"kubernetes.io/projected/5780a6e1-f323-4d22-bdb1-e937222424e7-kube-api-access-v5jbz\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.338881 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/de760bf0-c4f8-4281-980c-3abca5f7d0a7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.338881 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338864 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de760bf0-c4f8-4281-980c-3abca5f7d0a7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.338958 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338889 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de760bf0-c4f8-4281-980c-3abca5f7d0a7-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.338958 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5780a6e1-f323-4d22-bdb1-e937222424e7-metrics-client-ca\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.339031 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.338963 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.339031 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.339004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-textfile\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.339101 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.339027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-wtmp\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.339101 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.339068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vvks\" (UniqueName: \"kubernetes.io/projected/de760bf0-c4f8-4281-980c-3abca5f7d0a7-kube-api-access-8vvks\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.340454 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.340425 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de760bf0-c4f8-4281-980c-3abca5f7d0a7-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.342476 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.342419 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/de760bf0-c4f8-4281-980c-3abca5f7d0a7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.348304 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.348275 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de760bf0-c4f8-4281-980c-3abca5f7d0a7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.359973 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.359947 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vvks\" (UniqueName: \"kubernetes.io/projected/de760bf0-c4f8-4281-980c-3abca5f7d0a7-kube-api-access-8vvks\") pod \"openshift-state-metrics-65f78d5c66-6ksvb\" (UID: \"de760bf0-c4f8-4281-980c-3abca5f7d0a7\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.378270 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.378248 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.440098 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440073 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440211 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440115 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-textfile\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440211 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440139 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-wtmp\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440211 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-root\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440211 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440209 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-tls\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440412 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440234 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-sys\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440412 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440265 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440412 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440287 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5jbz\" (UniqueName: \"kubernetes.io/projected/5780a6e1-f323-4d22-bdb1-e937222424e7-kube-api-access-v5jbz\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440412 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5780a6e1-f323-4d22-bdb1-e937222424e7-metrics-client-ca\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440595 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440469 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-textfile\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440595 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:12:12.440579 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 11 13:12:12.440714 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:12:12.440675 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-tls podName:5780a6e1-f323-4d22-bdb1-e937222424e7 nodeName:}" failed. No retries permitted until 2026-06-11 13:12:12.940652709 +0000 UTC m=+96.183730523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-tls") pod "node-exporter-l6q47" (UID: "5780a6e1-f323-4d22-bdb1-e937222424e7") : secret "node-exporter-tls" not found Jun 11 13:12:12.440771 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440755 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440841 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440823 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-root\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440841 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440833 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5780a6e1-f323-4d22-bdb1-e937222424e7-metrics-client-ca\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440959 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440878 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-sys\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.440959 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.440940 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-wtmp\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.447847 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.447819 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.449251 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.449226 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5jbz\" (UniqueName: \"kubernetes.io/projected/5780a6e1-f323-4d22-bdb1-e937222424e7-kube-api-access-v5jbz\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.476532 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.476505 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.548261 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.548071 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" Jun 11 13:12:12.577526 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.577011 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.681717 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.681075 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.744944 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.744913 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb"] Jun 11 13:12:12.747184 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.747155 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.748704 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:12.748678 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde760bf0_c4f8_4281_980c_3abca5f7d0a7.slice/crio-7971bad14f4e1ee014391f5098484ec712f4f4380a54da97f9b3ab2764684a96 WatchSource:0}: Error finding container 7971bad14f4e1ee014391f5098484ec712f4f4380a54da97f9b3ab2764684a96: Status 404 returned error can't find the container with id 7971bad14f4e1ee014391f5098484ec712f4f4380a54da97f9b3ab2764684a96 Jun 11 13:12:12.804878 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.804822 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:12.944473 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.944443 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-tls\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:12.947036 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:12.947016 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5780a6e1-f323-4d22-bdb1-e937222424e7-node-exporter-tls\") pod \"node-exporter-l6q47\" (UID: \"5780a6e1-f323-4d22-bdb1-e937222424e7\") " pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:13.162676 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.162647 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:13.166438 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.166411 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l6q47" Jun 11 13:12:13.195104 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:13.195072 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5780a6e1_f323_4d22_bdb1_e937222424e7.slice/crio-3a3efe653e8c0aac161a219f695a88b469dc2b91340505f8fd1fcab95c9d613c WatchSource:0}: Error finding container 3a3efe653e8c0aac161a219f695a88b469dc2b91340505f8fd1fcab95c9d613c: Status 404 returned error can't find the container with id 3a3efe653e8c0aac161a219f695a88b469dc2b91340505f8fd1fcab95c9d613c Jun 11 13:12:13.261553 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.261518 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:13.341142 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.341097 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:13.345558 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.345522 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 13:12:13.374142 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.374113 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 13:12:13.374263 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.374231 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.377663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.377187 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-w8bdv\"" Jun 11 13:12:13.377663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.377203 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 11 13:12:13.377663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.377309 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 11 13:12:13.377663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.377322 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 11 13:12:13.377663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.377443 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 11 13:12:13.377663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.377443 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 11 13:12:13.377663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.377475 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 11 13:12:13.377663 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.377187 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 11 13:12:13.378135 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.378114 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 11 13:12:13.378298 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.378281 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 11 13:12:13.388342 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.388318 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:12:13.388434 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.388381 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:12:13.393537 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.393515 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:12:13.413385 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.413324 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.449816 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.449860 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/199ab944-4f6e-4f60-a67b-a6de7a59025b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.449921 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.449946 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-web-config\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.449992 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.450025 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfps2\" (UniqueName: \"kubernetes.io/projected/199ab944-4f6e-4f60-a67b-a6de7a59025b-kube-api-access-mfps2\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.450052 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/199ab944-4f6e-4f60-a67b-a6de7a59025b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.450078 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/199ab944-4f6e-4f60-a67b-a6de7a59025b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.450112 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-config-volume\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.450149 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/199ab944-4f6e-4f60-a67b-a6de7a59025b-config-out\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.450180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/199ab944-4f6e-4f60-a67b-a6de7a59025b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.450288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.451557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.451344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552094 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfps2\" (UniqueName: \"kubernetes.io/projected/199ab944-4f6e-4f60-a67b-a6de7a59025b-kube-api-access-mfps2\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552259 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552108 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/199ab944-4f6e-4f60-a67b-a6de7a59025b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552259 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552136 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/199ab944-4f6e-4f60-a67b-a6de7a59025b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552259 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552169 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-config-volume\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552259 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/199ab944-4f6e-4f60-a67b-a6de7a59025b-config-out\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552259 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/199ab944-4f6e-4f60-a67b-a6de7a59025b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552506 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552285 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552506 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552324 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552506 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552353 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552506 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552378 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/199ab944-4f6e-4f60-a67b-a6de7a59025b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552506 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552506 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552447 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-web-config\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.552506 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.552497 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.553215 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.553182 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/199ab944-4f6e-4f60-a67b-a6de7a59025b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.554060 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.553757 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/199ab944-4f6e-4f60-a67b-a6de7a59025b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.554613 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.554587 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/199ab944-4f6e-4f60-a67b-a6de7a59025b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.556449 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.555973 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/199ab944-4f6e-4f60-a67b-a6de7a59025b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.556449 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.556405 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.557658 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.557530 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/199ab944-4f6e-4f60-a67b-a6de7a59025b-config-out\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.557658 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.557581 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.557997 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.557975 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-config-volume\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.558090 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.558068 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.558330 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.558304 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.558995 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.558954 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.559386 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.559354 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/199ab944-4f6e-4f60-a67b-a6de7a59025b-web-config\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.563811 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.563789 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfps2\" (UniqueName: \"kubernetes.io/projected/199ab944-4f6e-4f60-a67b-a6de7a59025b-kube-api-access-mfps2\") pod \"alertmanager-main-0\" (UID: \"199ab944-4f6e-4f60-a67b-a6de7a59025b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.646749 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.646714 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:13.686918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.686859 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 13:12:13.748009 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.747981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:13.757925 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.757742 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" event={"ID":"de760bf0-c4f8-4281-980c-3abca5f7d0a7","Type":"ContainerStarted","Data":"a90ca8cda36e2bf47e87fc1f1279437f00d0eb54986633ff2f055473384e87b6"} Jun 11 13:12:13.757925 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.757808 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" event={"ID":"de760bf0-c4f8-4281-980c-3abca5f7d0a7","Type":"ContainerStarted","Data":"22ba09bcb6827e2e98d884eca62f87017bafa0b8d4e77970ccaf1b4b0ca16481"} Jun 11 13:12:13.757925 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.757840 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" event={"ID":"de760bf0-c4f8-4281-980c-3abca5f7d0a7","Type":"ContainerStarted","Data":"7971bad14f4e1ee014391f5098484ec712f4f4380a54da97f9b3ab2764684a96"} Jun 11 13:12:13.761434 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.761316 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6q47" event={"ID":"5780a6e1-f323-4d22-bdb1-e937222424e7","Type":"ContainerStarted","Data":"3a3efe653e8c0aac161a219f695a88b469dc2b91340505f8fd1fcab95c9d613c"} Jun 11 13:12:13.767061 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.766842 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:12:13.903514 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.903454 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:13.915696 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:13.915583 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 13:12:13.916760 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:13.916733 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod199ab944_4f6e_4f60_a67b_a6de7a59025b.slice/crio-ce523b7e39cb8c883a3e76b85a907f03797261158a46e14aff619b038064d15a WatchSource:0}: Error finding container ce523b7e39cb8c883a3e76b85a907f03797261158a46e14aff619b038064d15a: Status 404 returned error can't find the container with id ce523b7e39cb8c883a3e76b85a907f03797261158a46e14aff619b038064d15a Jun 11 13:12:14.003524 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:14.003427 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:14.640058 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:14.640007 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:14.741130 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:14.741102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:14.765373 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:14.765343 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"199ab944-4f6e-4f60-a67b-a6de7a59025b","Type":"ContainerStarted","Data":"ce523b7e39cb8c883a3e76b85a907f03797261158a46e14aff619b038064d15a"} Jun 11 13:12:14.841406 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:14.841380 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:15.172205 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.172166 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:15.267530 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.266314 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-766c45c974-tqzz4"] Jun 11 13:12:15.268818 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.268424 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:15.308322 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.307566 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-766c45c974-tqzz4"] Jun 11 13:12:15.308322 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.307730 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.310735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.310712 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-aopksb9uc4pep\"" Jun 11 13:12:15.310830 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.310769 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 11 13:12:15.310830 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.310801 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 11 13:12:15.311262 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.311087 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 11 13:12:15.311262 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.311099 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 11 13:12:15.311557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.311534 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kxks4\"" Jun 11 13:12:15.311765 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.311749 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 11 13:12:15.367958 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.367934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.368095 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.367982 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-tls\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.368095 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.368038 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.368095 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.368077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.368382 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.368106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.368382 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.368130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-grpc-tls\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.368382 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.368169 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a96a724d-26c4-469a-bdb2-7c235d235ee4-metrics-client-ca\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.368382 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.368193 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l85hj\" (UniqueName: \"kubernetes.io/projected/a96a724d-26c4-469a-bdb2-7c235d235ee4-kube-api-access-l85hj\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.441771 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.441738 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:15.469459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.468928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.469459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.468978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-tls\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.469459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.469024 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.469459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.469058 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.469459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.469089 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.469459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.469114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-grpc-tls\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.469459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.469150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a96a724d-26c4-469a-bdb2-7c235d235ee4-metrics-client-ca\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.469459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.469170 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l85hj\" (UniqueName: \"kubernetes.io/projected/a96a724d-26c4-469a-bdb2-7c235d235ee4-kube-api-access-l85hj\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.470421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.470350 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a96a724d-26c4-469a-bdb2-7c235d235ee4-metrics-client-ca\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.472594 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.472543 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.472702 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.472637 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-grpc-tls\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.472772 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.472750 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.472862 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.472836 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.474121 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.474097 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.474202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.474165 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a96a724d-26c4-469a-bdb2-7c235d235ee4-secret-thanos-querier-tls\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.476677 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.476658 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l85hj\" (UniqueName: \"kubernetes.io/projected/a96a724d-26c4-469a-bdb2-7c235d235ee4-kube-api-access-l85hj\") pod \"thanos-querier-766c45c974-tqzz4\" (UID: \"a96a724d-26c4-469a-bdb2-7c235d235ee4\") " pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.543037 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.543006 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:15.620127 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.620099 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:15.645767 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.645742 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-687799c7f-lq679" Jun 11 13:12:15.646179 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.645795 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-687799c7f-lq679" Jun 11 13:12:15.652526 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.652505 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-687799c7f-lq679" Jun 11 13:12:15.715389 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.715326 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6bbwn" Jun 11 13:12:15.771219 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.771175 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6q47" event={"ID":"5780a6e1-f323-4d22-bdb1-e937222424e7","Type":"ContainerStarted","Data":"530d116ec5da0b72f5ffebc3f3c97bc7c083b0e791b8486f06870c3f0ede4870"} Jun 11 13:12:15.773551 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.773522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" event={"ID":"de760bf0-c4f8-4281-980c-3abca5f7d0a7","Type":"ContainerStarted","Data":"12b0e8cf9df9d6262f805c4b8349674172de5f424fd4cecbd806cfa3086bcb4c"} Jun 11 13:12:15.777720 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.777699 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-687799c7f-lq679" Jun 11 13:12:15.825120 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.825066 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-6ksvb" podStartSLOduration=1.672365431 podStartE2EDuration="3.825045435s" totalImagesPullingTime="2.152680004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:15.823817541 +0000 UTC m=+99.066895393" watchObservedRunningTime="2026-06-11 13:12:15.825045435 +0000 UTC m=+99.068123269" Jun 11 13:12:15.838118 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.838093 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bf4d849d4-skq4q"] Jun 11 13:12:15.943258 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.943225 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:15.943820 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:15.943800 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-766c45c974-tqzz4"] Jun 11 13:12:15.946044 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:15.946009 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda96a724d_26c4_469a_bdb2_7c235d235ee4.slice/crio-953707342acb23c196aab4418cebdc5928f9d96969552b1b1c9fe53c57766f51 WatchSource:0}: Error finding container 953707342acb23c196aab4418cebdc5928f9d96969552b1b1c9fe53c57766f51: Status 404 returned error can't find the container with id 953707342acb23c196aab4418cebdc5928f9d96969552b1b1c9fe53c57766f51 Jun 11 13:12:16.041354 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.041289 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:16.141276 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.141257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:16.610571 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.610543 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-859cd447cd-wh2xg"] Jun 11 13:12:16.630247 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.630218 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-859cd447cd-wh2xg"] Jun 11 13:12:16.630395 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.630351 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.633525 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.633495 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dhovda3cbmjcf\"" Jun 11 13:12:16.634980 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.634603 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 11 13:12:16.634980 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.634711 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-g7r5x\"" Jun 11 13:12:16.634980 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.634723 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 11 13:12:16.634980 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.634863 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 11 13:12:16.634980 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.634938 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 11 13:12:16.679361 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.679286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj8gk\" (UniqueName: \"kubernetes.io/projected/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-kube-api-access-xj8gk\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.679734 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.679411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-client-ca-bundle\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.679734 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.679495 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-audit-log\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.679734 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.679519 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-metrics-server-audit-profiles\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.679734 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.679554 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.679734 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.679577 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-secret-metrics-server-client-certs\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.679734 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.679605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-secret-metrics-server-tls\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.780070 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.780038 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-audit-log\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.780219 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.780080 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-metrics-server-audit-profiles\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.780219 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.780110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.780219 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.780134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-secret-metrics-server-client-certs\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.780219 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.780176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-secret-metrics-server-tls\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.780219 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.780218 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj8gk\" (UniqueName: \"kubernetes.io/projected/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-kube-api-access-xj8gk\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.780458 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.780295 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-client-ca-bundle\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.781112 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.781089 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-audit-log\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.781778 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.781723 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.782696 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.781935 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-metrics-server-audit-profiles\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.782696 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.782547 2563 generic.go:356] "Generic (PLEG): container finished" podID="5780a6e1-f323-4d22-bdb1-e937222424e7" containerID="530d116ec5da0b72f5ffebc3f3c97bc7c083b0e791b8486f06870c3f0ede4870" exitCode=0 Jun 11 13:12:16.782696 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.782658 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6q47" event={"ID":"5780a6e1-f323-4d22-bdb1-e937222424e7","Type":"ContainerDied","Data":"530d116ec5da0b72f5ffebc3f3c97bc7c083b0e791b8486f06870c3f0ede4870"} Jun 11 13:12:16.783557 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.783538 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-secret-metrics-server-tls\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.783937 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.783916 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-client-ca-bundle\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.785205 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.785177 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-secret-metrics-server-client-certs\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.786555 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.786519 2563 generic.go:356] "Generic (PLEG): container finished" podID="199ab944-4f6e-4f60-a67b-a6de7a59025b" containerID="131013f8aaee4e25e468935b9f75852eced7b9f46e0cc99469243656ebd723d5" exitCode=0 Jun 11 13:12:16.786808 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.786789 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"199ab944-4f6e-4f60-a67b-a6de7a59025b","Type":"ContainerDied","Data":"131013f8aaee4e25e468935b9f75852eced7b9f46e0cc99469243656ebd723d5"} Jun 11 13:12:16.792950 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.792773 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" event={"ID":"a96a724d-26c4-469a-bdb2-7c235d235ee4","Type":"ContainerStarted","Data":"953707342acb23c196aab4418cebdc5928f9d96969552b1b1c9fe53c57766f51"} Jun 11 13:12:16.795018 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.794995 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj8gk\" (UniqueName: \"kubernetes.io/projected/d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73-kube-api-access-xj8gk\") pod \"metrics-server-859cd447cd-wh2xg\" (UID: \"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73\") " pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:16.943304 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:16.943285 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:17.005081 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.005045 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:17.017209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.016912 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr"] Jun 11 13:12:17.036286 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.036262 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr"] Jun 11 13:12:17.036543 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.036526 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:17.039918 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.039902 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-ztbf2\"" Jun 11 13:12:17.040036 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.039977 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 11 13:12:17.084517 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.084484 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b06cf1f-8154-42c0-99d1-2441a389f92a-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-c9vwr\" (UID: \"6b06cf1f-8154-42c0-99d1-2441a389f92a\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:17.084682 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.084603 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzrpg\" (UniqueName: \"kubernetes.io/projected/6b06cf1f-8154-42c0-99d1-2441a389f92a-kube-api-access-nzrpg\") pod \"monitoring-plugin-76b68cc874-c9vwr\" (UID: \"6b06cf1f-8154-42c0-99d1-2441a389f92a\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:17.108892 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.108838 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:17.128791 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.128746 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-859cd447cd-wh2xg"] Jun 11 13:12:17.131174 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:17.131147 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0314b1e_f6a4_4b8e_be3d_b8a6da1dfd73.slice/crio-f8b10c2896c9bb020c8e24944ff8b91b46c88e88c8f24a9aacd945ce394b5150 WatchSource:0}: Error finding container f8b10c2896c9bb020c8e24944ff8b91b46c88e88c8f24a9aacd945ce394b5150: Status 404 returned error can't find the container with id f8b10c2896c9bb020c8e24944ff8b91b46c88e88c8f24a9aacd945ce394b5150 Jun 11 13:12:17.185775 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.185744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzrpg\" (UniqueName: \"kubernetes.io/projected/6b06cf1f-8154-42c0-99d1-2441a389f92a-kube-api-access-nzrpg\") pod \"monitoring-plugin-76b68cc874-c9vwr\" (UID: \"6b06cf1f-8154-42c0-99d1-2441a389f92a\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:17.185885 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.185814 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b06cf1f-8154-42c0-99d1-2441a389f92a-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-c9vwr\" (UID: \"6b06cf1f-8154-42c0-99d1-2441a389f92a\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:17.185964 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:12:17.185950 2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 11 13:12:17.186024 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:12:17.186015 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b06cf1f-8154-42c0-99d1-2441a389f92a-monitoring-plugin-cert podName:6b06cf1f-8154-42c0-99d1-2441a389f92a nodeName:}" failed. No retries permitted until 2026-06-11 13:12:17.685994778 +0000 UTC m=+100.929072594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/6b06cf1f-8154-42c0-99d1-2441a389f92a-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-c9vwr" (UID: "6b06cf1f-8154-42c0-99d1-2441a389f92a") : secret "monitoring-plugin-cert" not found Jun 11 13:12:17.195880 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.195848 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzrpg\" (UniqueName: \"kubernetes.io/projected/6b06cf1f-8154-42c0-99d1-2441a389f92a-kube-api-access-nzrpg\") pod \"monitoring-plugin-76b68cc874-c9vwr\" (UID: \"6b06cf1f-8154-42c0-99d1-2441a389f92a\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:17.271099 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.271069 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:17.689820 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.689744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b06cf1f-8154-42c0-99d1-2441a389f92a-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-c9vwr\" (UID: \"6b06cf1f-8154-42c0-99d1-2441a389f92a\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:17.692775 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.692746 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b06cf1f-8154-42c0-99d1-2441a389f92a-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-c9vwr\" (UID: \"6b06cf1f-8154-42c0-99d1-2441a389f92a\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:17.800310 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.800224 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6q47" event={"ID":"5780a6e1-f323-4d22-bdb1-e937222424e7","Type":"ContainerStarted","Data":"28516999e944ae40c1ce505be0db8f832040982f1b60b2a80bdff4476c6e7179"} Jun 11 13:12:17.800310 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.800303 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6q47" event={"ID":"5780a6e1-f323-4d22-bdb1-e937222424e7","Type":"ContainerStarted","Data":"55bcdc42f032b2b25bb951ccdb6268f3c266be29167674232ebefb4bdcbee205"} Jun 11 13:12:17.801554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.801531 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" event={"ID":"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73","Type":"ContainerStarted","Data":"f8b10c2896c9bb020c8e24944ff8b91b46c88e88c8f24a9aacd945ce394b5150"} Jun 11 13:12:17.827654 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.827592 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l6q47" podStartSLOduration=3.751447889 podStartE2EDuration="5.827577414s" totalImagesPullingTime="1.972212435s" totalInitContainerRuntime="103.91709ms" isStatefulPod=true podCreationTimestamp="2026-06-11 13:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:17.825919597 +0000 UTC m=+101.068997433" watchObservedRunningTime="2026-06-11 13:12:17.827577414 +0000 UTC m=+101.070655246" Jun 11 13:12:17.830886 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.830859 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:17.947109 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.947036 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:17.949946 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:17.949922 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:18.098086 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.098043 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d59f8b96d-rpjzd"] Jun 11 13:12:18.111892 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.111849 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.112676 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.111863 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d59f8b96d-rpjzd"] Jun 11 13:12:18.195364 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.195329 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-oauth-serving-cert\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.195507 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.195411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-trusted-ca-bundle\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.195507 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.195444 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-oauth-config\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.195507 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.195487 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-service-ca\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.195652 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.195526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-serving-cert\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.195652 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.195581 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-console-config\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.195652 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.195638 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mz2t\" (UniqueName: \"kubernetes.io/projected/1757e543-d33a-40d0-aa07-b302fb96645b-kube-api-access-9mz2t\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.229695 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.229600 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr"] Jun 11 13:12:18.296255 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.296213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-oauth-serving-cert\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.296471 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.296306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-trusted-ca-bundle\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.296471 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.296341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-oauth-config\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.296471 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.296387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-service-ca\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.296471 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.296422 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-serving-cert\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.296471 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.296447 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-console-config\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.296764 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.296477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mz2t\" (UniqueName: \"kubernetes.io/projected/1757e543-d33a-40d0-aa07-b302fb96645b-kube-api-access-9mz2t\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.297089 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.297060 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-oauth-serving-cert\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.297304 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.297269 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-trusted-ca-bundle\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.297479 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.297445 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-service-ca\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.298004 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.297911 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-console-config\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.299722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.299698 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-oauth-config\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.299847 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.299830 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-serving-cert\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.310058 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.310014 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mz2t\" (UniqueName: \"kubernetes.io/projected/1757e543-d33a-40d0-aa07-b302fb96645b-kube-api-access-9mz2t\") pod \"console-7d59f8b96d-rpjzd\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.347853 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:18.347823 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b06cf1f_8154_42c0_99d1_2441a389f92a.slice/crio-9185d11688dd2df3c00c636ac24957f4f172ef2eb1c97fcf87f560d61979a238 WatchSource:0}: Error finding container 9185d11688dd2df3c00c636ac24957f4f172ef2eb1c97fcf87f560d61979a238: Status 404 returned error can't find the container with id 9185d11688dd2df3c00c636ac24957f4f172ef2eb1c97fcf87f560d61979a238 Jun 11 13:12:18.412528 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.412497 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:18.425217 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.425192 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:18.613527 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.613460 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d59f8b96d-rpjzd"] Jun 11 13:12:18.733399 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.733256 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:18.809001 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.808964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" event={"ID":"6b06cf1f-8154-42c0-99d1-2441a389f92a","Type":"ContainerStarted","Data":"9185d11688dd2df3c00c636ac24957f4f172ef2eb1c97fcf87f560d61979a238"} Jun 11 13:12:18.812097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.811953 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" event={"ID":"a96a724d-26c4-469a-bdb2-7c235d235ee4","Type":"ContainerStarted","Data":"eb39d36e02d5ca57085d030ae0ff22b84625ee26f16a9fe8dd41d23f48dde44b"} Jun 11 13:12:18.812097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.812038 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" event={"ID":"a96a724d-26c4-469a-bdb2-7c235d235ee4","Type":"ContainerStarted","Data":"7edea661f31e6a839b4f6e45b9470f536e3a6179ec6991798fe645bc66be942e"} Jun 11 13:12:18.831772 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.831729 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:18.932659 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:18.932567 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:19.119880 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:12:19.119814 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1757e543_d33a_40d0_aa07_b302fb96645b.slice/crio-ef6904f834e128ac1ed8d7f9bb8fa2a077b68cf28353b66559ba2f00a1a1692a WatchSource:0}: Error finding container ef6904f834e128ac1ed8d7f9bb8fa2a077b68cf28353b66559ba2f00a1a1692a: Status 404 returned error can't find the container with id ef6904f834e128ac1ed8d7f9bb8fa2a077b68cf28353b66559ba2f00a1a1692a Jun 11 13:12:19.818535 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:19.818507 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d59f8b96d-rpjzd" event={"ID":"1757e543-d33a-40d0-aa07-b302fb96645b","Type":"ContainerStarted","Data":"ef6904f834e128ac1ed8d7f9bb8fa2a077b68cf28353b66559ba2f00a1a1692a"} Jun 11 13:12:20.825669 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:20.825614 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"199ab944-4f6e-4f60-a67b-a6de7a59025b","Type":"ContainerStarted","Data":"ed90ca86b33b6d03f3575d7c7b231a0a14a0af4a6680d65f3be04ccadb235711"} Jun 11 13:12:20.826063 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:20.825727 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"199ab944-4f6e-4f60-a67b-a6de7a59025b","Type":"ContainerStarted","Data":"cd3acd17c7fb60a6233ca979d5b7417e2d1f17a2b9d9241b126997d5db056a9e"} Jun 11 13:12:20.827283 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:20.827255 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" event={"ID":"d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73","Type":"ContainerStarted","Data":"2b4e2c23c8240810f49aebcc5023a65490795dcd32a90c60a6664c3747c03d9e"} Jun 11 13:12:20.829571 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:20.829544 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" event={"ID":"a96a724d-26c4-469a-bdb2-7c235d235ee4","Type":"ContainerStarted","Data":"29e385f507348f07888454248092b6bcc811c2a6dd89d6201711c1b639ba7966"} Jun 11 13:12:20.831165 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:20.831144 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d59f8b96d-rpjzd" event={"ID":"1757e543-d33a-40d0-aa07-b302fb96645b","Type":"ContainerStarted","Data":"7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba"} Jun 11 13:12:20.845062 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:20.845024 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" podStartSLOduration=1.51566047 podStartE2EDuration="4.845010807s" totalImagesPullingTime="3.329350337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:20.843665533 +0000 UTC m=+104.086743385" watchObservedRunningTime="2026-06-11 13:12:20.845010807 +0000 UTC m=+104.088088639" Jun 11 13:12:20.860551 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:20.860512 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d59f8b96d-rpjzd" podStartSLOduration=2.860501253 podStartE2EDuration="2.860501253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:20.859670688 +0000 UTC m=+104.102748521" watchObservedRunningTime="2026-06-11 13:12:20.860501253 +0000 UTC m=+104.103579142" Jun 11 13:12:21.461234 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:21.461202 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:21.571089 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:21.571052 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:21.839908 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:21.839820 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"199ab944-4f6e-4f60-a67b-a6de7a59025b","Type":"ContainerStarted","Data":"f87360cdd54b369e83484984f16c2945e47317854188711c5cef4ee72f483504"} Jun 11 13:12:21.839908 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:21.839904 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"199ab944-4f6e-4f60-a67b-a6de7a59025b","Type":"ContainerStarted","Data":"1e0b09a1255acaaac828ee7b7089c813ecd25eda76e7eb57ae1492169312b155"} Jun 11 13:12:21.840403 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:21.839944 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"199ab944-4f6e-4f60-a67b-a6de7a59025b","Type":"ContainerStarted","Data":"24f73cda04a219fe6ce0ea1d4aeace16aec3bed5e39e36164a35648a89ec43c2"} Jun 11 13:12:21.841399 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:21.841371 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" event={"ID":"6b06cf1f-8154-42c0-99d1-2441a389f92a","Type":"ContainerStarted","Data":"da9789ea1243b0e021585ef8343bb8a1e55a0a949ddfb819976b653a0e7ba1c8"} Jun 11 13:12:21.862486 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:21.862445 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" podStartSLOduration=2.3438186979999998 podStartE2EDuration="4.862429393s" totalImagesPullingTime="2.518610695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:21.860934911 +0000 UTC m=+105.104012774" watchObservedRunningTime="2026-06-11 13:12:21.862429393 +0000 UTC m=+105.105507226" Jun 11 13:12:22.849039 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:22.849005 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"199ab944-4f6e-4f60-a67b-a6de7a59025b","Type":"ContainerStarted","Data":"98f2bda191fdbf358a0a0b723d41aa90264fda5fb4717847bdaa5033d02fe0ea"} Jun 11 13:12:22.852071 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:22.852046 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" event={"ID":"a96a724d-26c4-469a-bdb2-7c235d235ee4","Type":"ContainerStarted","Data":"d7f2da527730c5d10ee9858da12395292814d6996937eb854ec4550dbdcb021f"} Jun 11 13:12:22.852229 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:22.852118 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" event={"ID":"a96a724d-26c4-469a-bdb2-7c235d235ee4","Type":"ContainerStarted","Data":"5925de4573e237588c2e04075035cc247335896bb984bdf1fcb22edef465ce85"} Jun 11 13:12:22.852229 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:22.852145 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" event={"ID":"a96a724d-26c4-469a-bdb2-7c235d235ee4","Type":"ContainerStarted","Data":"0daadc3a2cbadddfee0e522cfe4a19003a5026d98a5ecdc98bfce56b19d98085"} Jun 11 13:12:22.852229 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:22.852191 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:22.857792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:22.857771 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-c9vwr" Jun 11 13:12:22.876959 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:22.876912 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.270113124 podStartE2EDuration="9.876893646s" totalImagesPullingTime="6.593843179s" totalInitContainerRuntime="12.937343ms" isStatefulPod=false podCreationTimestamp="2026-06-11 13:12:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:22.876495302 +0000 UTC m=+106.119573157" watchObservedRunningTime="2026-06-11 13:12:22.876893646 +0000 UTC m=+106.119971482" Jun 11 13:12:22.901344 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:22.901280 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" podStartSLOduration=4.138649651 podStartE2EDuration="7.901261339s" totalImagesPullingTime="3.762611688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:12:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:12:22.900761073 +0000 UTC m=+106.143838907" watchObservedRunningTime="2026-06-11 13:12:22.901261339 +0000 UTC m=+106.144339173" Jun 11 13:12:23.856584 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:23.856550 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:23.861656 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:23.861612 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-766c45c974-tqzz4" Jun 11 13:12:24.179492 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:24.179467 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" limit=1024 Jun 11 13:12:28.425803 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:28.425762 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:28.426201 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:28.425872 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:28.430337 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:28.430318 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:28.881550 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:28.881472 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:12:28.955905 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:28.955876 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-687799c7f-lq679"] Jun 11 13:12:36.722179 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:36.722143 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rw5vd" Jun 11 13:12:36.944117 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:36.944084 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:36.944285 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:36.944137 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:40.866004 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:40.865951 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-bf4d849d4-skq4q" podUID="8faa9812-3a93-4db1-a37d-e906a572f632" containerName="console" containerID="cri-o://2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116" gracePeriod=15 Jun 11 13:12:41.158421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.158398 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:12:41.289795 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.289772 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-console-config\") pod \"8faa9812-3a93-4db1-a37d-e906a572f632\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " Jun 11 13:12:41.289926 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.289808 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csf6t\" (UniqueName: \"kubernetes.io/projected/8faa9812-3a93-4db1-a37d-e906a572f632-kube-api-access-csf6t\") pod \"8faa9812-3a93-4db1-a37d-e906a572f632\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " Jun 11 13:12:41.289926 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.289849 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-oauth-config\") pod \"8faa9812-3a93-4db1-a37d-e906a572f632\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " Jun 11 13:12:41.289926 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.289873 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-serving-cert\") pod \"8faa9812-3a93-4db1-a37d-e906a572f632\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " Jun 11 13:12:41.290081 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.289933 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-service-ca\") pod \"8faa9812-3a93-4db1-a37d-e906a572f632\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " Jun 11 13:12:41.290081 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.289959 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-oauth-serving-cert\") pod \"8faa9812-3a93-4db1-a37d-e906a572f632\" (UID: \"8faa9812-3a93-4db1-a37d-e906a572f632\") " Jun 11 13:12:41.290194 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.290147 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-console-config" (OuterVolumeSpecName: "console-config") pod "8faa9812-3a93-4db1-a37d-e906a572f632" (UID: "8faa9812-3a93-4db1-a37d-e906a572f632"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:12:41.290253 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.290215 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-console-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:41.290552 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.290501 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-service-ca" (OuterVolumeSpecName: "service-ca") pod "8faa9812-3a93-4db1-a37d-e906a572f632" (UID: "8faa9812-3a93-4db1-a37d-e906a572f632"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:12:41.290552 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.290504 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8faa9812-3a93-4db1-a37d-e906a572f632" (UID: "8faa9812-3a93-4db1-a37d-e906a572f632"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:12:41.292288 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.292244 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8faa9812-3a93-4db1-a37d-e906a572f632" (UID: "8faa9812-3a93-4db1-a37d-e906a572f632"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:12:41.292288 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.292276 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8faa9812-3a93-4db1-a37d-e906a572f632-kube-api-access-csf6t" (OuterVolumeSpecName: "kube-api-access-csf6t") pod "8faa9812-3a93-4db1-a37d-e906a572f632" (UID: "8faa9812-3a93-4db1-a37d-e906a572f632"). InnerVolumeSpecName "kube-api-access-csf6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:12:41.292402 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.292309 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8faa9812-3a93-4db1-a37d-e906a572f632" (UID: "8faa9812-3a93-4db1-a37d-e906a572f632"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:12:41.390770 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.390715 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csf6t\" (UniqueName: \"kubernetes.io/projected/8faa9812-3a93-4db1-a37d-e906a572f632-kube-api-access-csf6t\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:41.390770 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.390738 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-oauth-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:41.390770 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.390747 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8faa9812-3a93-4db1-a37d-e906a572f632-console-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:41.390770 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.390755 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-service-ca\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:41.390770 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.390763 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8faa9812-3a93-4db1-a37d-e906a572f632-oauth-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:41.938290 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.938257 2563 generic.go:356] "Generic (PLEG): container finished" podID="8faa9812-3a93-4db1-a37d-e906a572f632" containerID="2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116" exitCode=0 Jun 11 13:12:41.938679 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.938303 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf4d849d4-skq4q" Jun 11 13:12:41.938679 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.938343 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf4d849d4-skq4q" event={"ID":"8faa9812-3a93-4db1-a37d-e906a572f632","Type":"ContainerDied","Data":"2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116"} Jun 11 13:12:41.938679 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.938400 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf4d849d4-skq4q" event={"ID":"8faa9812-3a93-4db1-a37d-e906a572f632","Type":"ContainerDied","Data":"3848c8de34e0530f75253059a1407a098ef6c4534c60717041b6a6a82d5ca0b6"} Jun 11 13:12:41.938679 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.938421 2563 scope.go:117] "RemoveContainer" containerID="2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116" Jun 11 13:12:41.947474 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.947456 2563 scope.go:117] "RemoveContainer" containerID="2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116" Jun 11 13:12:41.947756 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:12:41.947735 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116\": container with ID starting with 2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116 not found: ID does not exist" containerID="2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116" Jun 11 13:12:41.947810 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.947767 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116"} err="failed to get container status \"2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116\": rpc error: code = NotFound desc = could not find container \"2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116\": container with ID starting with 2bee17e38a1f2edc6439b1e619e46ecaf68704f487b6ff4f33a5a6cde4799116 not found: ID does not exist" Jun 11 13:12:41.959202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.959178 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bf4d849d4-skq4q"] Jun 11 13:12:41.962101 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:41.962083 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bf4d849d4-skq4q"] Jun 11 13:12:43.217137 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:43.217103 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8faa9812-3a93-4db1-a37d-e906a572f632" path="/var/lib/kubelet/pods/8faa9812-3a93-4db1-a37d-e906a572f632/volumes" Jun 11 13:12:53.975611 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:53.975571 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-687799c7f-lq679" podUID="6b9d2994-71b3-4437-b9ea-578f0e19acc8" containerName="console" containerID="cri-o://cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299" gracePeriod=15 Jun 11 13:12:54.268072 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.268047 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687799c7f-lq679" Jun 11 13:12:54.382281 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382258 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-oauth-config\") pod \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " Jun 11 13:12:54.382438 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382296 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-serving-cert\") pod \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " Jun 11 13:12:54.382438 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382329 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-config\") pod \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " Jun 11 13:12:54.382438 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382345 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-service-ca\") pod \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " Jun 11 13:12:54.382438 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382389 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-oauth-serving-cert\") pod \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " Jun 11 13:12:54.382438 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382410 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-trusted-ca-bundle\") pod \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " Jun 11 13:12:54.382438 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382426 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7wdk\" (UniqueName: \"kubernetes.io/projected/6b9d2994-71b3-4437-b9ea-578f0e19acc8-kube-api-access-l7wdk\") pod \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\" (UID: \"6b9d2994-71b3-4437-b9ea-578f0e19acc8\") " Jun 11 13:12:54.382832 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382790 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-config" (OuterVolumeSpecName: "console-config") pod "6b9d2994-71b3-4437-b9ea-578f0e19acc8" (UID: "6b9d2994-71b3-4437-b9ea-578f0e19acc8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:12:54.382945 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382859 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-service-ca" (OuterVolumeSpecName: "service-ca") pod "6b9d2994-71b3-4437-b9ea-578f0e19acc8" (UID: "6b9d2994-71b3-4437-b9ea-578f0e19acc8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:12:54.382945 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382855 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6b9d2994-71b3-4437-b9ea-578f0e19acc8" (UID: "6b9d2994-71b3-4437-b9ea-578f0e19acc8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:12:54.382945 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.382892 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6b9d2994-71b3-4437-b9ea-578f0e19acc8" (UID: "6b9d2994-71b3-4437-b9ea-578f0e19acc8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:12:54.384613 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.384587 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6b9d2994-71b3-4437-b9ea-578f0e19acc8" (UID: "6b9d2994-71b3-4437-b9ea-578f0e19acc8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:12:54.385154 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.385128 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6b9d2994-71b3-4437-b9ea-578f0e19acc8" (UID: "6b9d2994-71b3-4437-b9ea-578f0e19acc8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:12:54.385219 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.385149 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9d2994-71b3-4437-b9ea-578f0e19acc8-kube-api-access-l7wdk" (OuterVolumeSpecName: "kube-api-access-l7wdk") pod "6b9d2994-71b3-4437-b9ea-578f0e19acc8" (UID: "6b9d2994-71b3-4437-b9ea-578f0e19acc8"). InnerVolumeSpecName "kube-api-access-l7wdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:12:54.483559 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.483501 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-oauth-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:54.483559 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.483523 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:54.483559 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.483536 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-console-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:54.483559 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.483548 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-service-ca\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:54.483767 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.483561 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-oauth-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:54.483767 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.483573 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b9d2994-71b3-4437-b9ea-578f0e19acc8-trusted-ca-bundle\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:54.483767 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.483585 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7wdk\" (UniqueName: \"kubernetes.io/projected/6b9d2994-71b3-4437-b9ea-578f0e19acc8-kube-api-access-l7wdk\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:12:54.995226 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.995187 2563 generic.go:356] "Generic (PLEG): container finished" podID="6b9d2994-71b3-4437-b9ea-578f0e19acc8" containerID="cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299" exitCode=0 Jun 11 13:12:54.995699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.995235 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687799c7f-lq679" Jun 11 13:12:54.995699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.995266 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687799c7f-lq679" event={"ID":"6b9d2994-71b3-4437-b9ea-578f0e19acc8","Type":"ContainerDied","Data":"cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299"} Jun 11 13:12:54.995699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.995316 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687799c7f-lq679" event={"ID":"6b9d2994-71b3-4437-b9ea-578f0e19acc8","Type":"ContainerDied","Data":"57e8c59266a6ec4ea84f000e3553b8b04d5de31085a6a2433088704229adf1d1"} Jun 11 13:12:54.995699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:54.995337 2563 scope.go:117] "RemoveContainer" containerID="cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299" Jun 11 13:12:55.007494 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:55.007471 2563 scope.go:117] "RemoveContainer" containerID="cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299" Jun 11 13:12:55.007878 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:12:55.007852 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299\": container with ID starting with cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299 not found: ID does not exist" containerID="cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299" Jun 11 13:12:55.007950 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:55.007891 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299"} err="failed to get container status \"cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299\": rpc error: code = NotFound desc = could not find container \"cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299\": container with ID starting with cff6e3e0e8cfc2aa1b9a80cb8301c8b94e790d7eb7bb6ce5c2a78c3e454e9299 not found: ID does not exist" Jun 11 13:12:55.017328 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:55.017307 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-687799c7f-lq679"] Jun 11 13:12:55.021227 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:55.021205 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-687799c7f-lq679"] Jun 11 13:12:55.217128 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:55.217095 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b9d2994-71b3-4437-b9ea-578f0e19acc8" path="/var/lib/kubelet/pods/6b9d2994-71b3-4437-b9ea-578f0e19acc8/volumes" Jun 11 13:12:56.948521 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:56.948492 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:12:56.952380 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:12:56.952349 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-859cd447cd-wh2xg" Jun 11 13:13:36.510350 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.510157 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69cd66d5bf-mxsns"] Jun 11 13:13:36.510960 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.510938 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9d2994-71b3-4437-b9ea-578f0e19acc8" containerName="console" Jun 11 13:13:36.511043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.510963 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9d2994-71b3-4437-b9ea-578f0e19acc8" containerName="console" Jun 11 13:13:36.511043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.510977 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8faa9812-3a93-4db1-a37d-e906a572f632" containerName="console" Jun 11 13:13:36.511043 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.510984 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8faa9812-3a93-4db1-a37d-e906a572f632" containerName="console" Jun 11 13:13:36.511195 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.511067 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8faa9812-3a93-4db1-a37d-e906a572f632" containerName="console" Jun 11 13:13:36.511195 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.511084 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b9d2994-71b3-4437-b9ea-578f0e19acc8" containerName="console" Jun 11 13:13:36.513383 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.513362 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.517081 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.517049 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7jxts\"" Jun 11 13:13:36.517206 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.517051 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 11 13:13:36.517206 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.517172 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 11 13:13:36.517325 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.517291 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 11 13:13:36.517385 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.517334 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 11 13:13:36.518507 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.518477 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 11 13:13:36.523152 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.523126 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 11 13:13:36.524233 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.524211 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69cd66d5bf-mxsns"] Jun 11 13:13:36.598935 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.598907 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-secret-telemeter-client\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.599106 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.598945 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-telemeter-client-tls\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.599106 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.598971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-serving-certs-ca-bundle\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.599106 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.599018 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.599106 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.599041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-federate-client-tls\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.599106 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.599064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txjmp\" (UniqueName: \"kubernetes.io/projected/41ad60cd-584b-49c1-b7be-83b7abcdebf8-kube-api-access-txjmp\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.599318 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.599144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-metrics-client-ca\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.599318 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.599175 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.700286 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.700257 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-secret-telemeter-client\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.700418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.700295 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-telemeter-client-tls\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.700418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.700322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-serving-certs-ca-bundle\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.700418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.700346 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.700418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.700370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-federate-client-tls\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.700418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.700391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txjmp\" (UniqueName: \"kubernetes.io/projected/41ad60cd-584b-49c1-b7be-83b7abcdebf8-kube-api-access-txjmp\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.700704 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.700424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-metrics-client-ca\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.700704 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.700446 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.701360 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.701330 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-serving-certs-ca-bundle\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.701463 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.701395 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.701612 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.701591 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ad60cd-584b-49c1-b7be-83b7abcdebf8-metrics-client-ca\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.703072 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.703049 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-secret-telemeter-client\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.703224 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.703202 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-telemeter-client-tls\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.703650 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.703604 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.703695 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.703616 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/41ad60cd-584b-49c1-b7be-83b7abcdebf8-federate-client-tls\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.708942 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.708923 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txjmp\" (UniqueName: \"kubernetes.io/projected/41ad60cd-584b-49c1-b7be-83b7abcdebf8-kube-api-access-txjmp\") pod \"telemeter-client-69cd66d5bf-mxsns\" (UID: \"41ad60cd-584b-49c1-b7be-83b7abcdebf8\") " pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.826651 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.826570 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" Jun 11 13:13:36.954317 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:36.954293 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69cd66d5bf-mxsns"] Jun 11 13:13:36.955356 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:13:36.955330 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ad60cd_584b_49c1_b7be_83b7abcdebf8.slice/crio-6d354a0112d9ff2813beba1d16ca7a2300f2be83a9c342ee58b4885489514605 WatchSource:0}: Error finding container 6d354a0112d9ff2813beba1d16ca7a2300f2be83a9c342ee58b4885489514605: Status 404 returned error can't find the container with id 6d354a0112d9ff2813beba1d16ca7a2300f2be83a9c342ee58b4885489514605 Jun 11 13:13:37.197537 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:37.197508 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" event={"ID":"41ad60cd-584b-49c1-b7be-83b7abcdebf8","Type":"ContainerStarted","Data":"6d354a0112d9ff2813beba1d16ca7a2300f2be83a9c342ee58b4885489514605"} Jun 11 13:13:40.212596 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:40.212515 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" event={"ID":"41ad60cd-584b-49c1-b7be-83b7abcdebf8","Type":"ContainerStarted","Data":"d7b7daa3e3f12be871509b4333990453ebc9dc392824395ad299e110d90aa57c"} Jun 11 13:13:40.212596 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:40.212581 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" event={"ID":"41ad60cd-584b-49c1-b7be-83b7abcdebf8","Type":"ContainerStarted","Data":"194acb30dd1a9ec9ac58ac399585909a2953b44e255fd89e82be439ee72a86e4"} Jun 11 13:13:40.212966 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:40.212602 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" event={"ID":"41ad60cd-584b-49c1-b7be-83b7abcdebf8","Type":"ContainerStarted","Data":"06abf3989ff03a5b2da1f0798328755a0714cdf39f7a562e9a65b6de3c26549a"} Jun 11 13:13:40.234288 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:40.234235 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69cd66d5bf-mxsns" podStartSLOduration=1.250410517 podStartE2EDuration="4.234218487s" totalImagesPullingTime="2.98380797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:13:40.233846932 +0000 UTC m=+183.476924764" watchObservedRunningTime="2026-06-11 13:13:40.234218487 +0000 UTC m=+183.477296322" Jun 11 13:13:40.971438 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:40.971405 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b74586fdc-8qdj7"] Jun 11 13:13:40.973812 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:40.973790 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:40.984552 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:40.984526 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b74586fdc-8qdj7"] Jun 11 13:13:41.037024 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.036978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhbpc\" (UniqueName: \"kubernetes.io/projected/c44bfdfd-11a8-4d35-bc81-13b8878f0960-kube-api-access-qhbpc\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.037024 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.037026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-config\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.037209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.037053 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-oauth-serving-cert\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.037209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.037078 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-serving-cert\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.037209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.037098 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-service-ca\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.037209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.037114 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-trusted-ca-bundle\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.037209 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.037158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-oauth-config\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.138279 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.138252 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-oauth-serving-cert\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.138381 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.138288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-serving-cert\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.138381 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.138310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-service-ca\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.138381 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.138332 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-trusted-ca-bundle\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.138504 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.138459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-oauth-config\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.138576 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.138557 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhbpc\" (UniqueName: \"kubernetes.io/projected/c44bfdfd-11a8-4d35-bc81-13b8878f0960-kube-api-access-qhbpc\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.138651 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.138598 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-config\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.139102 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.139073 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-service-ca\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.139245 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.139228 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-config\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.139292 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.139249 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-oauth-serving-cert\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.139437 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.139417 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-trusted-ca-bundle\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.140874 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.140853 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-serving-cert\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.140962 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.140917 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-oauth-config\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.147702 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.147683 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhbpc\" (UniqueName: \"kubernetes.io/projected/c44bfdfd-11a8-4d35-bc81-13b8878f0960-kube-api-access-qhbpc\") pod \"console-5b74586fdc-8qdj7\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.283393 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.283338 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:41.431677 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:41.431649 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b74586fdc-8qdj7"] Jun 11 13:13:41.433488 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:13:41.433452 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44bfdfd_11a8_4d35_bc81_13b8878f0960.slice/crio-63bf723c483fa3fca52ae946d99eb534751acb4b9adb7559d58d736c28685f95 WatchSource:0}: Error finding container 63bf723c483fa3fca52ae946d99eb534751acb4b9adb7559d58d736c28685f95: Status 404 returned error can't find the container with id 63bf723c483fa3fca52ae946d99eb534751acb4b9adb7559d58d736c28685f95 Jun 11 13:13:42.223087 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:42.223055 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b74586fdc-8qdj7" event={"ID":"c44bfdfd-11a8-4d35-bc81-13b8878f0960","Type":"ContainerStarted","Data":"76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb"} Jun 11 13:13:42.223323 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:42.223100 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b74586fdc-8qdj7" event={"ID":"c44bfdfd-11a8-4d35-bc81-13b8878f0960","Type":"ContainerStarted","Data":"63bf723c483fa3fca52ae946d99eb534751acb4b9adb7559d58d736c28685f95"} Jun 11 13:13:42.240257 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:42.240209 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b74586fdc-8qdj7" podStartSLOduration=2.240194371 podStartE2EDuration="2.240194371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:13:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:13:42.238787225 +0000 UTC m=+185.481865058" watchObservedRunningTime="2026-06-11 13:13:42.240194371 +0000 UTC m=+185.483272257" Jun 11 13:13:51.283782 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:51.283748 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:51.284250 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:51.283813 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:51.288281 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:51.288260 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:52.269103 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.269070 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:13:52.315667 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.315635 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d59f8b96d-rpjzd"] Jun 11 13:13:52.420283 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.420258 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cb5954bbc-s7x9v"] Jun 11 13:13:52.422836 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.422820 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.434436 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.434414 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb5954bbc-s7x9v"] Jun 11 13:13:52.523198 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.523118 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-service-ca\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.523198 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.523165 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-trusted-ca-bundle\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.523375 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.523261 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-console-config\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.523375 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.523289 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-serving-cert\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.523375 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.523317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-oauth-serving-cert\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.523375 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.523337 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wfxq\" (UniqueName: \"kubernetes.io/projected/11daf53c-114c-4256-9ef0-649c017d7726-kube-api-access-2wfxq\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.523375 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.523368 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-oauth-config\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.624310 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.624281 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-service-ca\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.624483 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.624330 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-trusted-ca-bundle\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.624483 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.624386 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-console-config\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.624483 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.624423 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-serving-cert\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.624483 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.624448 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-oauth-serving-cert\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.624483 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.624475 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wfxq\" (UniqueName: \"kubernetes.io/projected/11daf53c-114c-4256-9ef0-649c017d7726-kube-api-access-2wfxq\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.624786 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.624517 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-oauth-config\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.625142 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.625113 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-service-ca\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.625347 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.625327 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-trusted-ca-bundle\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.625461 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.625440 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-console-config\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.625519 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.625448 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-oauth-serving-cert\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.627016 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.626985 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-serving-cert\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.627099 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.627076 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-oauth-config\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.632769 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.632748 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wfxq\" (UniqueName: \"kubernetes.io/projected/11daf53c-114c-4256-9ef0-649c017d7726-kube-api-access-2wfxq\") pod \"console-5cb5954bbc-s7x9v\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.732265 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.732240 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:13:52.863767 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:52.863745 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb5954bbc-s7x9v"] Jun 11 13:13:52.865584 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:13:52.865559 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11daf53c_114c_4256_9ef0_649c017d7726.slice/crio-b861eb3a0fd3d868ed58b4bc7e4632e599670a3384a9ab7099d99ab78c64de1a WatchSource:0}: Error finding container b861eb3a0fd3d868ed58b4bc7e4632e599670a3384a9ab7099d99ab78c64de1a: Status 404 returned error can't find the container with id b861eb3a0fd3d868ed58b4bc7e4632e599670a3384a9ab7099d99ab78c64de1a Jun 11 13:13:53.270531 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:53.270492 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb5954bbc-s7x9v" event={"ID":"11daf53c-114c-4256-9ef0-649c017d7726","Type":"ContainerStarted","Data":"d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490"} Jun 11 13:13:53.270732 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:53.270549 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb5954bbc-s7x9v" event={"ID":"11daf53c-114c-4256-9ef0-649c017d7726","Type":"ContainerStarted","Data":"b861eb3a0fd3d868ed58b4bc7e4632e599670a3384a9ab7099d99ab78c64de1a"} Jun 11 13:13:53.287767 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:13:53.287719 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cb5954bbc-s7x9v" podStartSLOduration=1.2877056279999999 podStartE2EDuration="1.287705628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:13:53.286427359 +0000 UTC m=+196.529505393" watchObservedRunningTime="2026-06-11 13:13:53.287705628 +0000 UTC m=+196.530783455" Jun 11 13:14:02.732498 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:02.732458 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:14:02.732948 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:02.732515 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:14:02.737322 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:02.737300 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:14:03.325552 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:03.325524 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:14:03.377934 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:03.377908 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b74586fdc-8qdj7"] Jun 11 13:14:17.335774 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.335709 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7d59f8b96d-rpjzd" podUID="1757e543-d33a-40d0-aa07-b302fb96645b" containerName="console" containerID="cri-o://7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba" gracePeriod=15 Jun 11 13:14:17.570793 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.570769 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:14:17.715666 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.715641 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-oauth-serving-cert\") pod \"1757e543-d33a-40d0-aa07-b302fb96645b\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " Jun 11 13:14:17.715778 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.715707 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mz2t\" (UniqueName: \"kubernetes.io/projected/1757e543-d33a-40d0-aa07-b302fb96645b-kube-api-access-9mz2t\") pod \"1757e543-d33a-40d0-aa07-b302fb96645b\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " Jun 11 13:14:17.715778 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.715771 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-service-ca\") pod \"1757e543-d33a-40d0-aa07-b302fb96645b\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " Jun 11 13:14:17.715855 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.715804 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-serving-cert\") pod \"1757e543-d33a-40d0-aa07-b302fb96645b\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " Jun 11 13:14:17.715904 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.715887 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-trusted-ca-bundle\") pod \"1757e543-d33a-40d0-aa07-b302fb96645b\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " Jun 11 13:14:17.715951 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.715937 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-oauth-config\") pod \"1757e543-d33a-40d0-aa07-b302fb96645b\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " Jun 11 13:14:17.716001 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.715968 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-console-config\") pod \"1757e543-d33a-40d0-aa07-b302fb96645b\" (UID: \"1757e543-d33a-40d0-aa07-b302fb96645b\") " Jun 11 13:14:17.716054 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.716025 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1757e543-d33a-40d0-aa07-b302fb96645b" (UID: "1757e543-d33a-40d0-aa07-b302fb96645b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:14:17.716255 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.716233 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-oauth-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:17.716369 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.716226 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-service-ca" (OuterVolumeSpecName: "service-ca") pod "1757e543-d33a-40d0-aa07-b302fb96645b" (UID: "1757e543-d33a-40d0-aa07-b302fb96645b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:14:17.716369 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.716305 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1757e543-d33a-40d0-aa07-b302fb96645b" (UID: "1757e543-d33a-40d0-aa07-b302fb96645b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:14:17.716447 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.716381 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-console-config" (OuterVolumeSpecName: "console-config") pod "1757e543-d33a-40d0-aa07-b302fb96645b" (UID: "1757e543-d33a-40d0-aa07-b302fb96645b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:14:17.718140 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.718112 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1757e543-d33a-40d0-aa07-b302fb96645b" (UID: "1757e543-d33a-40d0-aa07-b302fb96645b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:14:17.718476 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.718453 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1757e543-d33a-40d0-aa07-b302fb96645b" (UID: "1757e543-d33a-40d0-aa07-b302fb96645b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:14:17.718541 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.718478 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1757e543-d33a-40d0-aa07-b302fb96645b-kube-api-access-9mz2t" (OuterVolumeSpecName: "kube-api-access-9mz2t") pod "1757e543-d33a-40d0-aa07-b302fb96645b" (UID: "1757e543-d33a-40d0-aa07-b302fb96645b"). InnerVolumeSpecName "kube-api-access-9mz2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:14:17.817535 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.817510 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-service-ca\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:17.817535 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.817532 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:17.817699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.817541 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-trusted-ca-bundle\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:17.817699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.817550 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1757e543-d33a-40d0-aa07-b302fb96645b-console-oauth-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:17.817699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.817559 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1757e543-d33a-40d0-aa07-b302fb96645b-console-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:17.817699 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:17.817568 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mz2t\" (UniqueName: \"kubernetes.io/projected/1757e543-d33a-40d0-aa07-b302fb96645b-kube-api-access-9mz2t\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:18.386999 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.386963 2563 generic.go:356] "Generic (PLEG): container finished" podID="1757e543-d33a-40d0-aa07-b302fb96645b" containerID="7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba" exitCode=0 Jun 11 13:14:18.387466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.387015 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d59f8b96d-rpjzd" Jun 11 13:14:18.387466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.387031 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d59f8b96d-rpjzd" event={"ID":"1757e543-d33a-40d0-aa07-b302fb96645b","Type":"ContainerDied","Data":"7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba"} Jun 11 13:14:18.387466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.387081 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d59f8b96d-rpjzd" event={"ID":"1757e543-d33a-40d0-aa07-b302fb96645b","Type":"ContainerDied","Data":"ef6904f834e128ac1ed8d7f9bb8fa2a077b68cf28353b66559ba2f00a1a1692a"} Jun 11 13:14:18.387466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.387109 2563 scope.go:117] "RemoveContainer" containerID="7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba" Jun 11 13:14:18.395892 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.395874 2563 scope.go:117] "RemoveContainer" containerID="7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba" Jun 11 13:14:18.396169 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:14:18.396152 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba\": container with ID starting with 7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba not found: ID does not exist" containerID="7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba" Jun 11 13:14:18.396214 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.396176 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba"} err="failed to get container status \"7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba\": rpc error: code = NotFound desc = could not find container \"7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba\": container with ID starting with 7524580f54363d51439cdb09b7992406e515bb4719935c3dddff8cce116687ba not found: ID does not exist" Jun 11 13:14:18.418365 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.418336 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d59f8b96d-rpjzd"] Jun 11 13:14:18.429922 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:18.429901 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d59f8b96d-rpjzd"] Jun 11 13:14:19.217368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:19.217332 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1757e543-d33a-40d0-aa07-b302fb96645b" path="/var/lib/kubelet/pods/1757e543-d33a-40d0-aa07-b302fb96645b/volumes" Jun 11 13:14:28.398485 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.398441 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b74586fdc-8qdj7" podUID="c44bfdfd-11a8-4d35-bc81-13b8878f0960" containerName="console" containerID="cri-o://76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb" gracePeriod=15 Jun 11 13:14:28.661585 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.661557 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:14:28.714108 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714078 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhbpc\" (UniqueName: \"kubernetes.io/projected/c44bfdfd-11a8-4d35-bc81-13b8878f0960-kube-api-access-qhbpc\") pod \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " Jun 11 13:14:28.714256 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714120 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-oauth-serving-cert\") pod \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " Jun 11 13:14:28.714256 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714147 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-service-ca\") pod \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " Jun 11 13:14:28.714256 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714167 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-oauth-config\") pod \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " Jun 11 13:14:28.714421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714355 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-serving-cert\") pod \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " Jun 11 13:14:28.714476 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714462 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-config\") pod \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " Jun 11 13:14:28.714530 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714505 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-trusted-ca-bundle\") pod \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\" (UID: \"c44bfdfd-11a8-4d35-bc81-13b8878f0960\") " Jun 11 13:14:28.714580 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714520 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c44bfdfd-11a8-4d35-bc81-13b8878f0960" (UID: "c44bfdfd-11a8-4d35-bc81-13b8878f0960"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:14:28.714655 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714570 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-service-ca" (OuterVolumeSpecName: "service-ca") pod "c44bfdfd-11a8-4d35-bc81-13b8878f0960" (UID: "c44bfdfd-11a8-4d35-bc81-13b8878f0960"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:14:28.714850 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714793 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-oauth-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:28.714850 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714818 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-service-ca\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:28.714850 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714855 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-config" (OuterVolumeSpecName: "console-config") pod "c44bfdfd-11a8-4d35-bc81-13b8878f0960" (UID: "c44bfdfd-11a8-4d35-bc81-13b8878f0960"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:14:28.715220 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.714971 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c44bfdfd-11a8-4d35-bc81-13b8878f0960" (UID: "c44bfdfd-11a8-4d35-bc81-13b8878f0960"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:14:28.716400 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.716382 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c44bfdfd-11a8-4d35-bc81-13b8878f0960" (UID: "c44bfdfd-11a8-4d35-bc81-13b8878f0960"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:14:28.716913 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.716898 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c44bfdfd-11a8-4d35-bc81-13b8878f0960-kube-api-access-qhbpc" (OuterVolumeSpecName: "kube-api-access-qhbpc") pod "c44bfdfd-11a8-4d35-bc81-13b8878f0960" (UID: "c44bfdfd-11a8-4d35-bc81-13b8878f0960"). InnerVolumeSpecName "kube-api-access-qhbpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:14:28.716972 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.716910 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c44bfdfd-11a8-4d35-bc81-13b8878f0960" (UID: "c44bfdfd-11a8-4d35-bc81-13b8878f0960"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:14:28.816077 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.816049 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:28.816077 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.816078 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44bfdfd-11a8-4d35-bc81-13b8878f0960-trusted-ca-bundle\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:28.816232 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.816090 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhbpc\" (UniqueName: \"kubernetes.io/projected/c44bfdfd-11a8-4d35-bc81-13b8878f0960-kube-api-access-qhbpc\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:28.816232 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.816099 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-oauth-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:28.816232 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:28.816107 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44bfdfd-11a8-4d35-bc81-13b8878f0960-console-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:14:29.437205 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.437175 2563 generic.go:356] "Generic (PLEG): container finished" podID="c44bfdfd-11a8-4d35-bc81-13b8878f0960" containerID="76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb" exitCode=0 Jun 11 13:14:29.437604 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.437220 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b74586fdc-8qdj7" Jun 11 13:14:29.437604 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.437230 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b74586fdc-8qdj7" event={"ID":"c44bfdfd-11a8-4d35-bc81-13b8878f0960","Type":"ContainerDied","Data":"76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb"} Jun 11 13:14:29.437604 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.437279 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b74586fdc-8qdj7" event={"ID":"c44bfdfd-11a8-4d35-bc81-13b8878f0960","Type":"ContainerDied","Data":"63bf723c483fa3fca52ae946d99eb534751acb4b9adb7559d58d736c28685f95"} Jun 11 13:14:29.437604 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.437301 2563 scope.go:117] "RemoveContainer" containerID="76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb" Jun 11 13:14:29.445740 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.445716 2563 scope.go:117] "RemoveContainer" containerID="76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb" Jun 11 13:14:29.445991 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:14:29.445969 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb\": container with ID starting with 76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb not found: ID does not exist" containerID="76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb" Jun 11 13:14:29.446052 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.445999 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb"} err="failed to get container status \"76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb\": rpc error: code = NotFound desc = could not find container \"76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb\": container with ID starting with 76c98d18d7a5175a3118bb0c5a6c28f7009c8c38ff6853b34c120777802939cb not found: ID does not exist" Jun 11 13:14:29.454774 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.454754 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b74586fdc-8qdj7"] Jun 11 13:14:29.457458 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:29.457437 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b74586fdc-8qdj7"] Jun 11 13:14:31.217280 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:14:31.217236 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c44bfdfd-11a8-4d35-bc81-13b8878f0960" path="/var/lib/kubelet/pods/c44bfdfd-11a8-4d35-bc81-13b8878f0960/volumes" Jun 11 13:15:37.172406 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:15:37.172377 2563 kubelet.go:1648] "Image garbage collection succeeded" Jun 11 13:16:15.174027 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.173995 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn"] Jun 11 13:16:15.176646 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.174400 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1757e543-d33a-40d0-aa07-b302fb96645b" containerName="console" Jun 11 13:16:15.176646 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.174415 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1757e543-d33a-40d0-aa07-b302fb96645b" containerName="console" Jun 11 13:16:15.176646 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.174426 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c44bfdfd-11a8-4d35-bc81-13b8878f0960" containerName="console" Jun 11 13:16:15.176646 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.174432 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c44bfdfd-11a8-4d35-bc81-13b8878f0960" containerName="console" Jun 11 13:16:15.176646 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.174517 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c44bfdfd-11a8-4d35-bc81-13b8878f0960" containerName="console" Jun 11 13:16:15.176646 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.174525 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1757e543-d33a-40d0-aa07-b302fb96645b" containerName="console" Jun 11 13:16:15.177501 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.177483 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.180909 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.180885 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ppnnb\"" Jun 11 13:16:15.181028 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.180974 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 11 13:16:15.181125 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.181107 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 11 13:16:15.181375 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.181356 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 11 13:16:15.181542 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.181520 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 11 13:16:15.192417 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.192391 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn"] Jun 11 13:16:15.288123 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.288096 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/95965310-296e-4f56-b9e0-72f7690f2a3f-manifests\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.288267 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.288155 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95965310-296e-4f56-b9e0-72f7690f2a3f-tmp\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.288267 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.288190 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95965310-296e-4f56-b9e0-72f7690f2a3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.288267 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.288235 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/95965310-296e-4f56-b9e0-72f7690f2a3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.288401 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.288272 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhfzz\" (UniqueName: \"kubernetes.io/projected/95965310-296e-4f56-b9e0-72f7690f2a3f-kube-api-access-rhfzz\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.388833 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.388796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/95965310-296e-4f56-b9e0-72f7690f2a3f-manifests\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.388967 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.388875 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95965310-296e-4f56-b9e0-72f7690f2a3f-tmp\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.388967 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.388914 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95965310-296e-4f56-b9e0-72f7690f2a3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.388967 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.388952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/95965310-296e-4f56-b9e0-72f7690f2a3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.389097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.389017 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhfzz\" (UniqueName: \"kubernetes.io/projected/95965310-296e-4f56-b9e0-72f7690f2a3f-kube-api-access-rhfzz\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.389273 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.389224 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/95965310-296e-4f56-b9e0-72f7690f2a3f-manifests\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.391773 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.391748 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/95965310-296e-4f56-b9e0-72f7690f2a3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.391902 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.391788 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95965310-296e-4f56-b9e0-72f7690f2a3f-tmp\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.391902 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.391815 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95965310-296e-4f56-b9e0-72f7690f2a3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.397445 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.397413 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhfzz\" (UniqueName: \"kubernetes.io/projected/95965310-296e-4f56-b9e0-72f7690f2a3f-kube-api-access-rhfzz\") pod \"opendatahub-operator-controller-manager-7cdf5bf854-7nccn\" (UID: \"95965310-296e-4f56-b9e0-72f7690f2a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.488405 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.488346 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:15.637476 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.637447 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn"] Jun 11 13:16:15.640540 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:16:15.640514 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95965310_296e_4f56_b9e0_72f7690f2a3f.slice/crio-69a76702049c5d0c9a01d195f9532742bce8b46da58cfa05461cf8916e0d2819 WatchSource:0}: Error finding container 69a76702049c5d0c9a01d195f9532742bce8b46da58cfa05461cf8916e0d2819: Status 404 returned error can't find the container with id 69a76702049c5d0c9a01d195f9532742bce8b46da58cfa05461cf8916e0d2819 Jun 11 13:16:15.642278 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.642259 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 13:16:15.883632 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:15.883596 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" event={"ID":"95965310-296e-4f56-b9e0-72f7690f2a3f","Type":"ContainerStarted","Data":"69a76702049c5d0c9a01d195f9532742bce8b46da58cfa05461cf8916e0d2819"} Jun 11 13:16:19.904188 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:19.904151 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" event={"ID":"95965310-296e-4f56-b9e0-72f7690f2a3f","Type":"ContainerStarted","Data":"6796e906ef363b83138553465e57f4f5c199c2bb99535d95dea86fc41ec61917"} Jun 11 13:16:20.909788 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:20.909749 2563 generic.go:356] "Generic (PLEG): container finished" podID="95965310-296e-4f56-b9e0-72f7690f2a3f" containerID="6796e906ef363b83138553465e57f4f5c199c2bb99535d95dea86fc41ec61917" exitCode=0 Jun 11 13:16:20.910171 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:20.909840 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" event={"ID":"95965310-296e-4f56-b9e0-72f7690f2a3f","Type":"ContainerDied","Data":"6796e906ef363b83138553465e57f4f5c199c2bb99535d95dea86fc41ec61917"} Jun 11 13:16:21.915885 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:21.915847 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" event={"ID":"95965310-296e-4f56-b9e0-72f7690f2a3f","Type":"ContainerStarted","Data":"8c210d73d338aea905704a40eb8ea9ab15c9593efab08de9a5296bcdd18df136"} Jun 11 13:16:21.916366 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:21.915988 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:21.936656 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:21.936589 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" podStartSLOduration=1.8028177140000001 podStartE2EDuration="6.936575609s" totalImagesPullingTime="4.215037385s" totalInitContainerRuntime="918.72051ms" isStatefulPod=false podCreationTimestamp="2026-06-11 13:16:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:16:21.934438378 +0000 UTC m=+345.177516211" watchObservedRunningTime="2026-06-11 13:16:21.936575609 +0000 UTC m=+345.179653492" Jun 11 13:16:32.922612 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:32.922534 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cdf5bf854-7nccn" Jun 11 13:16:40.658440 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.658407 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9"] Jun 11 13:16:40.668603 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.668575 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.670163 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.670116 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9"] Jun 11 13:16:40.671421 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.671395 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 11 13:16:40.672611 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.672586 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 11 13:16:40.672719 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.672587 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 13:16:40.672807 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.672789 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fv74l\"" Jun 11 13:16:40.673189 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.673166 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 13:16:40.779797 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.779774 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9106ac49-59aa-4d34-8443-10d1d72aa60f-tmp\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.779898 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.779816 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhclg\" (UniqueName: \"kubernetes.io/projected/9106ac49-59aa-4d34-8443-10d1d72aa60f-kube-api-access-jhclg\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.779950 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.779904 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9106ac49-59aa-4d34-8443-10d1d72aa60f-tls-certs\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.880598 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.880563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9106ac49-59aa-4d34-8443-10d1d72aa60f-tmp\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.880692 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.880609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhclg\" (UniqueName: \"kubernetes.io/projected/9106ac49-59aa-4d34-8443-10d1d72aa60f-kube-api-access-jhclg\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.880692 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.880683 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9106ac49-59aa-4d34-8443-10d1d72aa60f-tls-certs\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.882866 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.882841 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9106ac49-59aa-4d34-8443-10d1d72aa60f-tmp\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.883195 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.883175 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9106ac49-59aa-4d34-8443-10d1d72aa60f-tls-certs\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.888881 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.888864 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhclg\" (UniqueName: \"kubernetes.io/projected/9106ac49-59aa-4d34-8443-10d1d72aa60f-kube-api-access-jhclg\") pod \"kube-auth-proxy-7f55b6cf49-4btt9\" (UID: \"9106ac49-59aa-4d34-8443-10d1d72aa60f\") " pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:40.979666 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:40.979600 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" Jun 11 13:16:41.134645 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:41.134593 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9"] Jun 11 13:16:41.137412 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:16:41.137380 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9106ac49_59aa_4d34_8443_10d1d72aa60f.slice/crio-8208b6163fa74de3fde4628c325c32820761888f1a5c8a19c02b8a825fb377fb WatchSource:0}: Error finding container 8208b6163fa74de3fde4628c325c32820761888f1a5c8a19c02b8a825fb377fb: Status 404 returned error can't find the container with id 8208b6163fa74de3fde4628c325c32820761888f1a5c8a19c02b8a825fb377fb Jun 11 13:16:42.004669 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:42.004607 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" event={"ID":"9106ac49-59aa-4d34-8443-10d1d72aa60f","Type":"ContainerStarted","Data":"8208b6163fa74de3fde4628c325c32820761888f1a5c8a19c02b8a825fb377fb"} Jun 11 13:16:45.022518 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.022481 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" event={"ID":"9106ac49-59aa-4d34-8443-10d1d72aa60f","Type":"ContainerStarted","Data":"10d0f41d5a37ee8c7ac58844b055237c956dc4506591c4ebd053e878b5baef61"} Jun 11 13:16:45.044544 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.044483 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f55b6cf49-4btt9" podStartSLOduration=1.995460645 podStartE2EDuration="5.044466504s" totalImagesPullingTime="3.049005859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:16:45.044125152 +0000 UTC m=+368.287203008" watchObservedRunningTime="2026-06-11 13:16:45.044466504 +0000 UTC m=+368.287544328" Jun 11 13:16:45.136601 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.136570 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-tdlg2"] Jun 11 13:16:45.143703 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.142298 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:45.146019 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.145993 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-78dd2\"" Jun 11 13:16:45.146323 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.146306 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 11 13:16:45.152836 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.152811 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-tdlg2"] Jun 11 13:16:45.175843 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.175820 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn"] Jun 11 13:16:45.178591 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.178574 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:45.181332 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.181310 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-j8jxx\"" Jun 11 13:16:45.181473 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.181456 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 11 13:16:45.188107 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.188087 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn"] Jun 11 13:16:45.223178 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.223117 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c4kh\" (UniqueName: \"kubernetes.io/projected/3638921d-7768-489d-988f-87e9a06e5b64-kube-api-access-6c4kh\") pod \"kserve-controller-manager-69fc866d76-tdlg2\" (UID: \"3638921d-7768-489d-988f-87e9a06e5b64\") " pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:45.223342 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.223226 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3638921d-7768-489d-988f-87e9a06e5b64-cert\") pod \"kserve-controller-manager-69fc866d76-tdlg2\" (UID: \"3638921d-7768-489d-988f-87e9a06e5b64\") " pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:45.324050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.323971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82a9297a-b0c2-43f5-bab1-753963757bd4-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-z2hqn\" (UID: \"82a9297a-b0c2-43f5-bab1-753963757bd4\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:45.324050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.324045 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c4kh\" (UniqueName: \"kubernetes.io/projected/3638921d-7768-489d-988f-87e9a06e5b64-kube-api-access-6c4kh\") pod \"kserve-controller-manager-69fc866d76-tdlg2\" (UID: \"3638921d-7768-489d-988f-87e9a06e5b64\") " pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:45.324235 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.324084 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3638921d-7768-489d-988f-87e9a06e5b64-cert\") pod \"kserve-controller-manager-69fc866d76-tdlg2\" (UID: \"3638921d-7768-489d-988f-87e9a06e5b64\") " pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:45.324235 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.324113 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st5zf\" (UniqueName: \"kubernetes.io/projected/82a9297a-b0c2-43f5-bab1-753963757bd4-kube-api-access-st5zf\") pod \"llmisvc-controller-manager-864bfd7bc9-z2hqn\" (UID: \"82a9297a-b0c2-43f5-bab1-753963757bd4\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:45.324235 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:16:45.324214 2563 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 11 13:16:45.324385 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:16:45.324298 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3638921d-7768-489d-988f-87e9a06e5b64-cert podName:3638921d-7768-489d-988f-87e9a06e5b64 nodeName:}" failed. No retries permitted until 2026-06-11 13:16:45.824275049 +0000 UTC m=+369.067352873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3638921d-7768-489d-988f-87e9a06e5b64-cert") pod "kserve-controller-manager-69fc866d76-tdlg2" (UID: "3638921d-7768-489d-988f-87e9a06e5b64") : secret "kserve-webhook-server-cert" not found Jun 11 13:16:45.338979 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.338948 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c4kh\" (UniqueName: \"kubernetes.io/projected/3638921d-7768-489d-988f-87e9a06e5b64-kube-api-access-6c4kh\") pod \"kserve-controller-manager-69fc866d76-tdlg2\" (UID: \"3638921d-7768-489d-988f-87e9a06e5b64\") " pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:45.425525 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.425498 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zf\" (UniqueName: \"kubernetes.io/projected/82a9297a-b0c2-43f5-bab1-753963757bd4-kube-api-access-st5zf\") pod \"llmisvc-controller-manager-864bfd7bc9-z2hqn\" (UID: \"82a9297a-b0c2-43f5-bab1-753963757bd4\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:45.425664 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.425545 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82a9297a-b0c2-43f5-bab1-753963757bd4-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-z2hqn\" (UID: \"82a9297a-b0c2-43f5-bab1-753963757bd4\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:45.427940 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.427919 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82a9297a-b0c2-43f5-bab1-753963757bd4-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-z2hqn\" (UID: \"82a9297a-b0c2-43f5-bab1-753963757bd4\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:45.438163 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.438143 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st5zf\" (UniqueName: \"kubernetes.io/projected/82a9297a-b0c2-43f5-bab1-753963757bd4-kube-api-access-st5zf\") pod \"llmisvc-controller-manager-864bfd7bc9-z2hqn\" (UID: \"82a9297a-b0c2-43f5-bab1-753963757bd4\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:45.489696 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.489669 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:45.622361 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.622334 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn"] Jun 11 13:16:45.624115 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:16:45.624076 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod82a9297a_b0c2_43f5_bab1_753963757bd4.slice/crio-778d0c3b7e6b81884f7de7f1e36c64dec57223cb24486e526b722440e79f58e3 WatchSource:0}: Error finding container 778d0c3b7e6b81884f7de7f1e36c64dec57223cb24486e526b722440e79f58e3: Status 404 returned error can't find the container with id 778d0c3b7e6b81884f7de7f1e36c64dec57223cb24486e526b722440e79f58e3 Jun 11 13:16:45.830266 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.830238 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3638921d-7768-489d-988f-87e9a06e5b64-cert\") pod \"kserve-controller-manager-69fc866d76-tdlg2\" (UID: \"3638921d-7768-489d-988f-87e9a06e5b64\") " pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:45.832614 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:45.832597 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3638921d-7768-489d-988f-87e9a06e5b64-cert\") pod \"kserve-controller-manager-69fc866d76-tdlg2\" (UID: \"3638921d-7768-489d-988f-87e9a06e5b64\") " pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:46.027816 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:46.027781 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" event={"ID":"82a9297a-b0c2-43f5-bab1-753963757bd4","Type":"ContainerStarted","Data":"778d0c3b7e6b81884f7de7f1e36c64dec57223cb24486e526b722440e79f58e3"} Jun 11 13:16:46.055479 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:46.055456 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:46.187909 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:46.187885 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-tdlg2"] Jun 11 13:16:46.215100 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:16:46.215071 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3638921d_7768_489d_988f_87e9a06e5b64.slice/crio-c6cb4ae649eb1fe094b001eaa59f8f1609ad005b0ab6d6d0bc44e8f35be6e9de WatchSource:0}: Error finding container c6cb4ae649eb1fe094b001eaa59f8f1609ad005b0ab6d6d0bc44e8f35be6e9de: Status 404 returned error can't find the container with id c6cb4ae649eb1fe094b001eaa59f8f1609ad005b0ab6d6d0bc44e8f35be6e9de Jun 11 13:16:47.035599 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:47.035530 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" event={"ID":"3638921d-7768-489d-988f-87e9a06e5b64","Type":"ContainerStarted","Data":"c6cb4ae649eb1fe094b001eaa59f8f1609ad005b0ab6d6d0bc44e8f35be6e9de"} Jun 11 13:16:50.051907 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:50.051864 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" event={"ID":"3638921d-7768-489d-988f-87e9a06e5b64","Type":"ContainerStarted","Data":"4ac3e84091ebcc8feba38da5140be732320d6c2ae89f7f6f270a996e9622e6dd"} Jun 11 13:16:50.052325 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:50.051994 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:16:50.053360 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:50.053336 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" event={"ID":"82a9297a-b0c2-43f5-bab1-753963757bd4","Type":"ContainerStarted","Data":"e262534549876ada054bbae504b1ffde116da0f3cdda02e1976d7fcf7cdff88a"} Jun 11 13:16:50.053419 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:50.053402 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:16:50.081821 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:50.081783 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" podStartSLOduration=2.2135816090000002 podStartE2EDuration="5.081770462s" totalImagesPullingTime="2.868188853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:16:50.079175538 +0000 UTC m=+373.322253370" watchObservedRunningTime="2026-06-11 13:16:50.081770462 +0000 UTC m=+373.324848295" Jun 11 13:16:50.096446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:50.096408 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" podStartSLOduration=1.633424097 podStartE2EDuration="5.096395322s" totalImagesPullingTime="3.462971225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:16:50.095678207 +0000 UTC m=+373.338756039" watchObservedRunningTime="2026-06-11 13:16:50.096395322 +0000 UTC m=+373.339473199" Jun 11 13:16:57.207970 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.207935 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-68c659989f-ghq4z"] Jun 11 13:16:57.211081 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.211055 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" Jun 11 13:16:57.214500 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.214475 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6n2nd\"" Jun 11 13:16:57.215464 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.215444 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 11 13:16:57.225920 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.225896 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-68c659989f-ghq4z"] Jun 11 13:16:57.228288 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.228263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a9326b61-159d-4b6e-9e32-9a015a1e2182-redirect-config\") pod \"dashboard-redirect-68c659989f-ghq4z\" (UID: \"a9326b61-159d-4b6e-9e32-9a015a1e2182\") " pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" Jun 11 13:16:57.228393 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.228304 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7dk7\" (UniqueName: \"kubernetes.io/projected/a9326b61-159d-4b6e-9e32-9a015a1e2182-kube-api-access-m7dk7\") pod \"dashboard-redirect-68c659989f-ghq4z\" (UID: \"a9326b61-159d-4b6e-9e32-9a015a1e2182\") " pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" Jun 11 13:16:57.329488 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.329461 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7dk7\" (UniqueName: \"kubernetes.io/projected/a9326b61-159d-4b6e-9e32-9a015a1e2182-kube-api-access-m7dk7\") pod \"dashboard-redirect-68c659989f-ghq4z\" (UID: \"a9326b61-159d-4b6e-9e32-9a015a1e2182\") " pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" Jun 11 13:16:57.329652 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.329588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a9326b61-159d-4b6e-9e32-9a015a1e2182-redirect-config\") pod \"dashboard-redirect-68c659989f-ghq4z\" (UID: \"a9326b61-159d-4b6e-9e32-9a015a1e2182\") " pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" Jun 11 13:16:57.330250 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.330226 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a9326b61-159d-4b6e-9e32-9a015a1e2182-redirect-config\") pod \"dashboard-redirect-68c659989f-ghq4z\" (UID: \"a9326b61-159d-4b6e-9e32-9a015a1e2182\") " pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" Jun 11 13:16:57.339114 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.339093 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7dk7\" (UniqueName: \"kubernetes.io/projected/a9326b61-159d-4b6e-9e32-9a015a1e2182-kube-api-access-m7dk7\") pod \"dashboard-redirect-68c659989f-ghq4z\" (UID: \"a9326b61-159d-4b6e-9e32-9a015a1e2182\") " pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" Jun 11 13:16:57.522121 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.522046 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" Jun 11 13:16:57.653359 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:57.653330 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-68c659989f-ghq4z"] Jun 11 13:16:57.655009 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:16:57.654983 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9326b61_159d_4b6e_9e32_9a015a1e2182.slice/crio-f1f8a94340e94fb3ee1d0d7cc5f0b53f92022db2b0e7ef0aa71486cc5384cd2f WatchSource:0}: Error finding container f1f8a94340e94fb3ee1d0d7cc5f0b53f92022db2b0e7ef0aa71486cc5384cd2f: Status 404 returned error can't find the container with id f1f8a94340e94fb3ee1d0d7cc5f0b53f92022db2b0e7ef0aa71486cc5384cd2f Jun 11 13:16:58.088768 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:58.088732 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" event={"ID":"a9326b61-159d-4b6e-9e32-9a015a1e2182","Type":"ContainerStarted","Data":"f1f8a94340e94fb3ee1d0d7cc5f0b53f92022db2b0e7ef0aa71486cc5384cd2f"} Jun 11 13:16:59.084789 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.084759 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-ks74c"] Jun 11 13:16:59.088007 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.087986 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:16:59.093303 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.093280 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 11 13:16:59.093485 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.093464 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-kbsnn\"" Jun 11 13:16:59.093485 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.093480 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 11 13:16:59.101849 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.101827 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-ks74c"] Jun 11 13:16:59.143345 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.143318 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9-operator-config\") pod \"servicemesh-operator3-57f65f65fb-ks74c\" (UID: \"bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:16:59.143448 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.143426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g4rr\" (UniqueName: \"kubernetes.io/projected/bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9-kube-api-access-9g4rr\") pod \"servicemesh-operator3-57f65f65fb-ks74c\" (UID: \"bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:16:59.244050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.244013 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9-operator-config\") pod \"servicemesh-operator3-57f65f65fb-ks74c\" (UID: \"bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:16:59.244188 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.244095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g4rr\" (UniqueName: \"kubernetes.io/projected/bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9-kube-api-access-9g4rr\") pod \"servicemesh-operator3-57f65f65fb-ks74c\" (UID: \"bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:16:59.247149 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.247120 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9-operator-config\") pod \"servicemesh-operator3-57f65f65fb-ks74c\" (UID: \"bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:16:59.252968 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.252942 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g4rr\" (UniqueName: \"kubernetes.io/projected/bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9-kube-api-access-9g4rr\") pod \"servicemesh-operator3-57f65f65fb-ks74c\" (UID: \"bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:16:59.400194 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.400122 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:16:59.557952 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:16:59.557917 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-ks74c"] Jun 11 13:16:59.562191 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:16:59.562165 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc8c0b6_0fa9_414a_b70c_fe7fa762bcc9.slice/crio-0731f594b639958d48c503c07638e68c32be299884491acbc44a66d708e5af6b WatchSource:0}: Error finding container 0731f594b639958d48c503c07638e68c32be299884491acbc44a66d708e5af6b: Status 404 returned error can't find the container with id 0731f594b639958d48c503c07638e68c32be299884491acbc44a66d708e5af6b Jun 11 13:17:00.108050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:00.108005 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" event={"ID":"bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9","Type":"ContainerStarted","Data":"0731f594b639958d48c503c07638e68c32be299884491acbc44a66d708e5af6b"} Jun 11 13:17:03.124875 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:03.124841 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" event={"ID":"a9326b61-159d-4b6e-9e32-9a015a1e2182","Type":"ContainerStarted","Data":"6aac57b4b3502eb24c59637362aec796811262e155329cde60aeb242a6f51e79"} Jun 11 13:17:03.141563 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:03.141518 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-68c659989f-ghq4z" podStartSLOduration=1.583096278 podStartE2EDuration="6.141505184s" totalImagesPullingTime="4.558408906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:17:03.140227595 +0000 UTC m=+386.383305465" watchObservedRunningTime="2026-06-11 13:17:03.141505184 +0000 UTC m=+386.384583014" Jun 11 13:17:05.137368 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:05.137329 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" event={"ID":"bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9","Type":"ContainerStarted","Data":"a1ea98505a07596298eb35c93618e18a8469061c8dbeee25372d34daf6a99f9b"} Jun 11 13:17:05.137826 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:05.137480 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:17:05.159784 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:05.159737 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" podStartSLOduration=1.463080934 podStartE2EDuration="6.159722727s" totalImagesPullingTime="4.696641793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:17:05.157271175 +0000 UTC m=+388.400349007" watchObservedRunningTime="2026-06-11 13:17:05.159722727 +0000 UTC m=+388.402800559" Jun 11 13:17:09.987688 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:09.987654 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9"] Jun 11 13:17:09.991666 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:09.991648 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:09.994293 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:09.994270 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 11 13:17:09.994293 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:09.994288 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 11 13:17:09.994467 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:09.994325 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 11 13:17:09.994467 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:09.994290 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hw65f\"" Jun 11 13:17:09.994467 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:09.994289 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 11 13:17:10.003445 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.003422 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9"] Jun 11 13:17:10.030518 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.030494 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.030653 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.030549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/691275c6-2abb-47a8-ab09-dccc20061027-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.030706 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.030684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.030745 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.030713 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.030786 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.030764 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/691275c6-2abb-47a8-ab09-dccc20061027-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.030827 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.030795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/691275c6-2abb-47a8-ab09-dccc20061027-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.030871 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.030832 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ljqh\" (UniqueName: \"kubernetes.io/projected/691275c6-2abb-47a8-ab09-dccc20061027-kube-api-access-4ljqh\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.132030 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.131990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/691275c6-2abb-47a8-ab09-dccc20061027-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.132170 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.132082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.132170 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.132110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.132170 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.132161 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/691275c6-2abb-47a8-ab09-dccc20061027-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.132321 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.132268 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/691275c6-2abb-47a8-ab09-dccc20061027-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.132372 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.132321 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ljqh\" (UniqueName: \"kubernetes.io/projected/691275c6-2abb-47a8-ab09-dccc20061027-kube-api-access-4ljqh\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.132487 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.132461 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.132716 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.132690 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/691275c6-2abb-47a8-ab09-dccc20061027-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.134698 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.134670 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.134790 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.134679 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.135225 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.135205 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/691275c6-2abb-47a8-ab09-dccc20061027-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.135429 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.135411 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/691275c6-2abb-47a8-ab09-dccc20061027-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.140395 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.140366 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ljqh\" (UniqueName: \"kubernetes.io/projected/691275c6-2abb-47a8-ab09-dccc20061027-kube-api-access-4ljqh\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.140501 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.140483 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/691275c6-2abb-47a8-ab09-dccc20061027-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-z57z9\" (UID: \"691275c6-2abb-47a8-ab09-dccc20061027\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.301648 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.301551 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:10.436941 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:10.436859 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9"] Jun 11 13:17:10.440036 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:17:10.440001 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691275c6_2abb_47a8_ab09_dccc20061027.slice/crio-a573497c45652f703f7838bb2df501450b7c09f674764bced9c3574b6ab6c7fb WatchSource:0}: Error finding container a573497c45652f703f7838bb2df501450b7c09f674764bced9c3574b6ab6c7fb: Status 404 returned error can't find the container with id a573497c45652f703f7838bb2df501450b7c09f674764bced9c3574b6ab6c7fb Jun 11 13:17:11.171613 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:11.171578 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" event={"ID":"691275c6-2abb-47a8-ab09-dccc20061027","Type":"ContainerStarted","Data":"a573497c45652f703f7838bb2df501450b7c09f674764bced9c3574b6ab6c7fb"} Jun 11 13:17:13.756151 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:13.756113 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 11 13:17:13.756422 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:13.756189 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 11 13:17:14.189395 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:14.189357 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" event={"ID":"691275c6-2abb-47a8-ab09-dccc20061027","Type":"ContainerStarted","Data":"0433c2c74e0f61e1a3533480c473bee399c65826c709cd2d3052080e13dae4d3"} Jun 11 13:17:14.189583 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:14.189571 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:14.191306 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:14.191277 2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-z57z9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 11 13:17:14.191435 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:14.191332 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" podUID="691275c6-2abb-47a8-ab09-dccc20061027" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 13:17:14.208954 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:14.208906 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" podStartSLOduration=1.8954762 podStartE2EDuration="5.208892119s" totalImagesPullingTime="3.313415919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:17:14.208097646 +0000 UTC m=+397.451175492" watchObservedRunningTime="2026-06-11 13:17:14.208892119 +0000 UTC m=+397.451969951" Jun 11 13:17:15.195147 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:15.195120 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-z57z9" Jun 11 13:17:16.143871 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:16.143798 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-ks74c" Jun 11 13:17:21.060669 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:21.060636 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2hqn" Jun 11 13:17:21.063884 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:17:21.063863 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-tdlg2" Jun 11 13:18:08.272703 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.268155 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht"] Jun 11 13:18:08.277167 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.277140 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" Jun 11 13:18:08.279999 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.279970 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 11 13:18:08.279999 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.279994 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ddznf\"" Jun 11 13:18:08.280217 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.280090 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 13:18:08.280359 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.280334 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 13:18:08.280999 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.280982 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht"] Jun 11 13:18:08.393233 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.393204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s8x4\" (UniqueName: \"kubernetes.io/projected/3772cbd4-6e28-411d-a432-8e2597e59863-kube-api-access-7s8x4\") pod \"dns-operator-controller-manager-864dd78c4c-c8lht\" (UID: \"3772cbd4-6e28-411d-a432-8e2597e59863\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" Jun 11 13:18:08.493764 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.493735 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s8x4\" (UniqueName: \"kubernetes.io/projected/3772cbd4-6e28-411d-a432-8e2597e59863-kube-api-access-7s8x4\") pod \"dns-operator-controller-manager-864dd78c4c-c8lht\" (UID: \"3772cbd4-6e28-411d-a432-8e2597e59863\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" Jun 11 13:18:08.503344 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.503314 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s8x4\" (UniqueName: \"kubernetes.io/projected/3772cbd4-6e28-411d-a432-8e2597e59863-kube-api-access-7s8x4\") pod \"dns-operator-controller-manager-864dd78c4c-c8lht\" (UID: \"3772cbd4-6e28-411d-a432-8e2597e59863\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" Jun 11 13:18:08.589016 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.588968 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" Jun 11 13:18:08.720434 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:08.720412 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht"] Jun 11 13:18:08.722522 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:18:08.722496 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3772cbd4_6e28_411d_a432_8e2597e59863.slice/crio-5f6203042fe2d3025871d12781e29af530292a515365a7794ab11ea0831928c6 WatchSource:0}: Error finding container 5f6203042fe2d3025871d12781e29af530292a515365a7794ab11ea0831928c6: Status 404 returned error can't find the container with id 5f6203042fe2d3025871d12781e29af530292a515365a7794ab11ea0831928c6 Jun 11 13:18:09.454570 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:09.454539 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" event={"ID":"3772cbd4-6e28-411d-a432-8e2597e59863","Type":"ContainerStarted","Data":"5f6203042fe2d3025871d12781e29af530292a515365a7794ab11ea0831928c6"} Jun 11 13:18:11.465352 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:11.465316 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" event={"ID":"3772cbd4-6e28-411d-a432-8e2597e59863","Type":"ContainerStarted","Data":"5e7938ad1b3b71e888d424908add3a6ab56c8ffff139b0e62ea22ab2309abc8b"} Jun 11 13:18:11.465711 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:11.465450 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" Jun 11 13:18:11.492480 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:11.492436 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" podStartSLOduration=1.18719624 podStartE2EDuration="3.492422536s" totalImagesPullingTime="2.305226296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:18:11.490096095 +0000 UTC m=+454.733173953" watchObservedRunningTime="2026-06-11 13:18:11.492422536 +0000 UTC m=+454.735500368" Jun 11 13:18:12.431072 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.431034 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz"] Jun 11 13:18:12.434743 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.434723 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:12.438098 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.438076 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nbdpr\"" Jun 11 13:18:12.447553 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.447529 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz"] Jun 11 13:18:12.530548 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.530527 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2mhl\" (UniqueName: \"kubernetes.io/projected/41083cf7-3c1f-4785-b1b9-2312ff935a60-kube-api-access-p2mhl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" (UID: \"41083cf7-3c1f-4785-b1b9-2312ff935a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:12.530861 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.530612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41083cf7-3c1f-4785-b1b9-2312ff935a60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" (UID: \"41083cf7-3c1f-4785-b1b9-2312ff935a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:12.631509 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.631485 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2mhl\" (UniqueName: \"kubernetes.io/projected/41083cf7-3c1f-4785-b1b9-2312ff935a60-kube-api-access-p2mhl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" (UID: \"41083cf7-3c1f-4785-b1b9-2312ff935a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:12.631656 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.631558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41083cf7-3c1f-4785-b1b9-2312ff935a60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" (UID: \"41083cf7-3c1f-4785-b1b9-2312ff935a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:12.631874 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.631855 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41083cf7-3c1f-4785-b1b9-2312ff935a60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" (UID: \"41083cf7-3c1f-4785-b1b9-2312ff935a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:12.641225 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.641199 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2mhl\" (UniqueName: \"kubernetes.io/projected/41083cf7-3c1f-4785-b1b9-2312ff935a60-kube-api-access-p2mhl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" (UID: \"41083cf7-3c1f-4785-b1b9-2312ff935a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:12.746138 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.746086 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:12.881524 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:12.881502 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz"] Jun 11 13:18:12.883683 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:18:12.883655 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41083cf7_3c1f_4785_b1b9_2312ff935a60.slice/crio-b671f9383981bfd93d92f005f25c8d8cf8a870f072be89a91eae50cb9c173b6a WatchSource:0}: Error finding container b671f9383981bfd93d92f005f25c8d8cf8a870f072be89a91eae50cb9c173b6a: Status 404 returned error can't find the container with id b671f9383981bfd93d92f005f25c8d8cf8a870f072be89a91eae50cb9c173b6a Jun 11 13:18:13.478100 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:13.477573 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" event={"ID":"41083cf7-3c1f-4785-b1b9-2312ff935a60","Type":"ContainerStarted","Data":"b671f9383981bfd93d92f005f25c8d8cf8a870f072be89a91eae50cb9c173b6a"} Jun 11 13:18:20.519684 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:20.519647 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" event={"ID":"41083cf7-3c1f-4785-b1b9-2312ff935a60","Type":"ContainerStarted","Data":"5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed"} Jun 11 13:18:20.520063 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:20.519708 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:20.548797 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:20.548752 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" podStartSLOduration=1.429507734 podStartE2EDuration="8.548738723s" totalImagesPullingTime="7.119230989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:18:20.546333792 +0000 UTC m=+463.789411626" watchObservedRunningTime="2026-06-11 13:18:20.548738723 +0000 UTC m=+463.791816555" Jun 11 13:18:22.473317 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:22.473288 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8lht" Jun 11 13:18:31.526433 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:31.526395 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:32.635955 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.635922 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk"] Jun 11 13:18:32.639431 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.639415 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:32.653032 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.653011 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk"] Jun 11 13:18:32.738665 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.738643 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk"] Jun 11 13:18:32.738857 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:32.738839 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-mswrh], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" podUID="884af4e3-967a-4af6-9393-cc3f1e2fa707" Jun 11 13:18:32.795592 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.795568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mswrh\" (UniqueName: \"kubernetes.io/projected/884af4e3-967a-4af6-9393-cc3f1e2fa707-kube-api-access-mswrh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" (UID: \"884af4e3-967a-4af6-9393-cc3f1e2fa707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:32.795710 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.795602 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/884af4e3-967a-4af6-9393-cc3f1e2fa707-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" (UID: \"884af4e3-967a-4af6-9393-cc3f1e2fa707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:32.897038 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.896965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mswrh\" (UniqueName: \"kubernetes.io/projected/884af4e3-967a-4af6-9393-cc3f1e2fa707-kube-api-access-mswrh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" (UID: \"884af4e3-967a-4af6-9393-cc3f1e2fa707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:32.897038 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.897010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/884af4e3-967a-4af6-9393-cc3f1e2fa707-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" (UID: \"884af4e3-967a-4af6-9393-cc3f1e2fa707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:32.897322 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.897302 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/884af4e3-967a-4af6-9393-cc3f1e2fa707-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" (UID: \"884af4e3-967a-4af6-9393-cc3f1e2fa707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:32.918815 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:32.918793 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mswrh\" (UniqueName: \"kubernetes.io/projected/884af4e3-967a-4af6-9393-cc3f1e2fa707-kube-api-access-mswrh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" (UID: \"884af4e3-967a-4af6-9393-cc3f1e2fa707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:33.224466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.224440 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk"] Jun 11 13:18:33.236019 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.235997 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk"] Jun 11 13:18:33.246930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.246908 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz"] Jun 11 13:18:33.247091 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.247068 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" containerName="manager" containerID="cri-o://5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed" gracePeriod=2 Jun 11 13:18:33.272115 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.272088 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz"] Jun 11 13:18:33.273182 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.273153 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"41083cf7-3c1f-4785-b1b9-2312ff935a60\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\": pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.281568 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.281542 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd"] Jun 11 13:18:33.282132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.282113 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" containerName="manager" Jun 11 13:18:33.282132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.282132 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" containerName="manager" Jun 11 13:18:33.282364 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.282212 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" containerName="manager" Jun 11 13:18:33.285087 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.285070 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:33.287848 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.287823 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.300533 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.300511 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd"] Jun 11 13:18:33.310834 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.310811 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8"] Jun 11 13:18:33.314306 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.314291 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" Jun 11 13:18:33.317241 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.317218 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jvsz7\"" Jun 11 13:18:33.326429 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.326397 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8"] Jun 11 13:18:33.340436 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.340409 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.400652 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.400595 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3760e24f-6a55-4805-ab9e-739ab04bbe86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x4chd\" (UID: \"3760e24f-6a55-4805-ab9e-739ab04bbe86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:33.400811 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.400765 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mj4f\" (UniqueName: \"kubernetes.io/projected/3760e24f-6a55-4805-ab9e-739ab04bbe86-kube-api-access-4mj4f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x4chd\" (UID: \"3760e24f-6a55-4805-ab9e-739ab04bbe86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:33.400890 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.400828 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snvrm\" (UniqueName: \"kubernetes.io/projected/8e928376-fb70-442a-9d86-e3d86555d2ad-kube-api-access-snvrm\") pod \"limitador-operator-controller-manager-859d7b78bd-68bp8\" (UID: \"8e928376-fb70-442a-9d86-e3d86555d2ad\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" Jun 11 13:18:33.486001 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.485947 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.488378 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.488354 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.501981 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.501952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3760e24f-6a55-4805-ab9e-739ab04bbe86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x4chd\" (UID: \"3760e24f-6a55-4805-ab9e-739ab04bbe86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:33.502104 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.502088 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mj4f\" (UniqueName: \"kubernetes.io/projected/3760e24f-6a55-4805-ab9e-739ab04bbe86-kube-api-access-4mj4f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x4chd\" (UID: \"3760e24f-6a55-4805-ab9e-739ab04bbe86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:33.502170 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.502155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snvrm\" (UniqueName: \"kubernetes.io/projected/8e928376-fb70-442a-9d86-e3d86555d2ad-kube-api-access-snvrm\") pod \"limitador-operator-controller-manager-859d7b78bd-68bp8\" (UID: \"8e928376-fb70-442a-9d86-e3d86555d2ad\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" Jun 11 13:18:33.502314 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.502292 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3760e24f-6a55-4805-ab9e-739ab04bbe86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x4chd\" (UID: \"3760e24f-6a55-4805-ab9e-739ab04bbe86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:33.514180 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.514158 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mj4f\" (UniqueName: \"kubernetes.io/projected/3760e24f-6a55-4805-ab9e-739ab04bbe86-kube-api-access-4mj4f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x4chd\" (UID: \"3760e24f-6a55-4805-ab9e-739ab04bbe86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:33.514459 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.514442 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snvrm\" (UniqueName: \"kubernetes.io/projected/8e928376-fb70-442a-9d86-e3d86555d2ad-kube-api-access-snvrm\") pod \"limitador-operator-controller-manager-859d7b78bd-68bp8\" (UID: \"8e928376-fb70-442a-9d86-e3d86555d2ad\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" Jun 11 13:18:33.592777 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.592745 2563 generic.go:356] "Generic (PLEG): container finished" podID="41083cf7-3c1f-4785-b1b9-2312ff935a60" containerID="5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed" exitCode=0 Jun 11 13:18:33.592877 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.592787 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.592877 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.592854 2563 scope.go:117] "RemoveContainer" containerID="5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed" Jun 11 13:18:33.593240 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.592900 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:33.595358 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.595323 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.599275 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.599251 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.600177 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.600155 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:33.601462 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.601440 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="884af4e3-967a-4af6-9393-cc3f1e2fa707" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:33.601723 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.601707 2563 scope.go:117] "RemoveContainer" containerID="5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed" Jun 11 13:18:33.601966 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.601947 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed\": container with ID starting with 5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed not found: ID does not exist" containerID="5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed" Jun 11 13:18:33.602033 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.601977 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed"} err="failed to get container status \"5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed\": rpc error: code = NotFound desc = could not find container \"5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed\": container with ID starting with 5475fcdcdd11e3c0cc542deadd1a6b6afd5140193494da7259d0670b44336bed not found: ID does not exist" Jun 11 13:18:33.603196 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.603181 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41083cf7-3c1f-4785-b1b9-2312ff935a60-extensions-socket-volume\") pod \"41083cf7-3c1f-4785-b1b9-2312ff935a60\" (UID: \"41083cf7-3c1f-4785-b1b9-2312ff935a60\") " Jun 11 13:18:33.603251 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.603237 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2mhl\" (UniqueName: \"kubernetes.io/projected/41083cf7-3c1f-4785-b1b9-2312ff935a60-kube-api-access-p2mhl\") pod \"41083cf7-3c1f-4785-b1b9-2312ff935a60\" (UID: \"41083cf7-3c1f-4785-b1b9-2312ff935a60\") " Jun 11 13:18:33.603593 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.603573 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41083cf7-3c1f-4785-b1b9-2312ff935a60-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "41083cf7-3c1f-4785-b1b9-2312ff935a60" (UID: "41083cf7-3c1f-4785-b1b9-2312ff935a60"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 13:18:33.603796 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.603773 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="884af4e3-967a-4af6-9393-cc3f1e2fa707" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:33.605237 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.605218 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41083cf7-3c1f-4785-b1b9-2312ff935a60-kube-api-access-p2mhl" (OuterVolumeSpecName: "kube-api-access-p2mhl") pod "41083cf7-3c1f-4785-b1b9-2312ff935a60" (UID: "41083cf7-3c1f-4785-b1b9-2312ff935a60"). InnerVolumeSpecName "kube-api-access-p2mhl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:18:33.607206 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.607183 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:33.629519 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.629498 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:33.637581 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.637564 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" Jun 11 13:18:33.704724 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.704691 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mswrh\" (UniqueName: \"kubernetes.io/projected/884af4e3-967a-4af6-9393-cc3f1e2fa707-kube-api-access-mswrh\") pod \"884af4e3-967a-4af6-9393-cc3f1e2fa707\" (UID: \"884af4e3-967a-4af6-9393-cc3f1e2fa707\") " Jun 11 13:18:33.704874 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.704789 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/884af4e3-967a-4af6-9393-cc3f1e2fa707-extensions-socket-volume\") pod \"884af4e3-967a-4af6-9393-cc3f1e2fa707\" (UID: \"884af4e3-967a-4af6-9393-cc3f1e2fa707\") " Jun 11 13:18:33.705068 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.705037 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41083cf7-3c1f-4785-b1b9-2312ff935a60-extensions-socket-volume\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:18:33.705141 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.705075 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2mhl\" (UniqueName: \"kubernetes.io/projected/41083cf7-3c1f-4785-b1b9-2312ff935a60-kube-api-access-p2mhl\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:18:33.705193 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.705175 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/884af4e3-967a-4af6-9393-cc3f1e2fa707-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "884af4e3-967a-4af6-9393-cc3f1e2fa707" (UID: "884af4e3-967a-4af6-9393-cc3f1e2fa707"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 13:18:33.707137 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.707104 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/884af4e3-967a-4af6-9393-cc3f1e2fa707-kube-api-access-mswrh" (OuterVolumeSpecName: "kube-api-access-mswrh") pod "884af4e3-967a-4af6-9393-cc3f1e2fa707" (UID: "884af4e3-967a-4af6-9393-cc3f1e2fa707"). InnerVolumeSpecName "kube-api-access-mswrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:18:33.787968 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.787940 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd"] Jun 11 13:18:33.790937 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:18:33.790910 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3760e24f_6a55_4805_ab9e_739ab04bbe86.slice/crio-b339aeee293c349ee5858aef86ede33223f96878aad6594459bcde6a93e704f4 WatchSource:0}: Error finding container b339aeee293c349ee5858aef86ede33223f96878aad6594459bcde6a93e704f4: Status 404 returned error can't find the container with id b339aeee293c349ee5858aef86ede33223f96878aad6594459bcde6a93e704f4 Jun 11 13:18:33.794565 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.794545 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8"] Jun 11 13:18:33.796132 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:18:33.796102 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e928376_fb70_442a_9d86_e3d86555d2ad.slice/crio-cae8e62188f8986eb6faf3ac62155601535957ca00fa5dc02392540cbb5d7aba WatchSource:0}: Error finding container cae8e62188f8986eb6faf3ac62155601535957ca00fa5dc02392540cbb5d7aba: Status 404 returned error can't find the container with id cae8e62188f8986eb6faf3ac62155601535957ca00fa5dc02392540cbb5d7aba Jun 11 13:18:33.806339 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.806314 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/884af4e3-967a-4af6-9393-cc3f1e2fa707-extensions-socket-volume\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:18:33.806339 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:33.806335 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mswrh\" (UniqueName: \"kubernetes.io/projected/884af4e3-967a-4af6-9393-cc3f1e2fa707-kube-api-access-mswrh\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:18:33.902639 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.902590 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="884af4e3-967a-4af6-9393-cc3f1e2fa707" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:33.904712 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:33.904691 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:34.599479 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:34.599446 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" event={"ID":"3760e24f-6a55-4805-ab9e-739ab04bbe86","Type":"ContainerStarted","Data":"5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef"} Jun 11 13:18:34.599687 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:34.599500 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:34.599687 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:34.599517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" event={"ID":"3760e24f-6a55-4805-ab9e-739ab04bbe86","Type":"ContainerStarted","Data":"b339aeee293c349ee5858aef86ede33223f96878aad6594459bcde6a93e704f4"} Jun 11 13:18:34.600790 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:34.600761 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" event={"ID":"8e928376-fb70-442a-9d86-e3d86555d2ad","Type":"ContainerStarted","Data":"cae8e62188f8986eb6faf3ac62155601535957ca00fa5dc02392540cbb5d7aba"} Jun 11 13:18:34.601606 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:34.601586 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:34.602110 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:34.602085 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="884af4e3-967a-4af6-9393-cc3f1e2fa707" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:34.604315 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:34.604291 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:34.626085 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:34.626038 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" podStartSLOduration=1.626023507 podStartE2EDuration="1.626023507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:18:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:18:34.624073061 +0000 UTC m=+477.867150910" watchObservedRunningTime="2026-06-11 13:18:34.626023507 +0000 UTC m=+477.869101340" Jun 11 13:18:34.626277 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:34.626252 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9n2nz\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9n2nz" Jun 11 13:18:34.628335 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:34.628314 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lmrbk\" is forbidden: User \"system:node:ip-10-0-141-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-252.ec2.internal' and this object" podUID="884af4e3-967a-4af6-9393-cc3f1e2fa707" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lmrbk" Jun 11 13:18:35.218140 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:35.218104 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41083cf7-3c1f-4785-b1b9-2312ff935a60" path="/var/lib/kubelet/pods/41083cf7-3c1f-4785-b1b9-2312ff935a60/volumes" Jun 11 13:18:35.218487 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:35.218450 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="884af4e3-967a-4af6-9393-cc3f1e2fa707" path="/var/lib/kubelet/pods/884af4e3-967a-4af6-9393-cc3f1e2fa707/volumes" Jun 11 13:18:35.608474 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:35.608378 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" event={"ID":"8e928376-fb70-442a-9d86-e3d86555d2ad","Type":"ContainerStarted","Data":"6e324502dc7ca0023858ea7981da39be5078eedcb0074c97dacf091d161a416c"} Jun 11 13:18:35.608647 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:35.608604 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" Jun 11 13:18:35.626835 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:35.626784 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" podStartSLOduration=1.1912012459999999 podStartE2EDuration="2.626770237s" totalImagesPullingTime="1.435568991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:18:35.624884098 +0000 UTC m=+478.867961958" watchObservedRunningTime="2026-06-11 13:18:35.626770237 +0000 UTC m=+478.869848070" Jun 11 13:18:45.610591 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:45.610560 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:46.616790 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:46.616753 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68bp8" Jun 11 13:18:51.452900 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.452863 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd"] Jun 11 13:18:51.453282 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.453034 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" podUID="3760e24f-6a55-4805-ab9e-739ab04bbe86" containerName="manager" containerID="cri-o://5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef" gracePeriod=10 Jun 11 13:18:51.713578 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.713500 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7"] Jun 11 13:18:51.719591 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.719566 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:51.730285 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.730262 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7"] Jun 11 13:18:51.852527 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.852497 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzpkm\" (UniqueName: \"kubernetes.io/projected/d3ece720-508b-47fb-8434-15a19734ae7a-kube-api-access-pzpkm\") pod \"kuadrant-operator-controller-manager-86878c769f-8z5k7\" (UID: \"d3ece720-508b-47fb-8434-15a19734ae7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:51.852613 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.852582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ece720-508b-47fb-8434-15a19734ae7a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8z5k7\" (UID: \"d3ece720-508b-47fb-8434-15a19734ae7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:51.953357 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.953327 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ece720-508b-47fb-8434-15a19734ae7a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8z5k7\" (UID: \"d3ece720-508b-47fb-8434-15a19734ae7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:51.953491 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.953429 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzpkm\" (UniqueName: \"kubernetes.io/projected/d3ece720-508b-47fb-8434-15a19734ae7a-kube-api-access-pzpkm\") pod \"kuadrant-operator-controller-manager-86878c769f-8z5k7\" (UID: \"d3ece720-508b-47fb-8434-15a19734ae7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:51.953738 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.953720 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ece720-508b-47fb-8434-15a19734ae7a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8z5k7\" (UID: \"d3ece720-508b-47fb-8434-15a19734ae7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:51.962668 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.962639 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzpkm\" (UniqueName: \"kubernetes.io/projected/d3ece720-508b-47fb-8434-15a19734ae7a-kube-api-access-pzpkm\") pod \"kuadrant-operator-controller-manager-86878c769f-8z5k7\" (UID: \"d3ece720-508b-47fb-8434-15a19734ae7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:51.998356 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:51.998335 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:52.030964 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.030923 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:52.155108 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.155081 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mj4f\" (UniqueName: \"kubernetes.io/projected/3760e24f-6a55-4805-ab9e-739ab04bbe86-kube-api-access-4mj4f\") pod \"3760e24f-6a55-4805-ab9e-739ab04bbe86\" (UID: \"3760e24f-6a55-4805-ab9e-739ab04bbe86\") " Jun 11 13:18:52.155259 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.155143 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3760e24f-6a55-4805-ab9e-739ab04bbe86-extensions-socket-volume\") pod \"3760e24f-6a55-4805-ab9e-739ab04bbe86\" (UID: \"3760e24f-6a55-4805-ab9e-739ab04bbe86\") " Jun 11 13:18:52.155520 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.155496 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3760e24f-6a55-4805-ab9e-739ab04bbe86-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3760e24f-6a55-4805-ab9e-739ab04bbe86" (UID: "3760e24f-6a55-4805-ab9e-739ab04bbe86"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 13:18:52.157093 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.157073 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3760e24f-6a55-4805-ab9e-739ab04bbe86-kube-api-access-4mj4f" (OuterVolumeSpecName: "kube-api-access-4mj4f") pod "3760e24f-6a55-4805-ab9e-739ab04bbe86" (UID: "3760e24f-6a55-4805-ab9e-739ab04bbe86"). InnerVolumeSpecName "kube-api-access-4mj4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:18:52.166771 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.166749 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7"] Jun 11 13:18:52.169259 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:18:52.169238 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ece720_508b_47fb_8434_15a19734ae7a.slice/crio-f3cb32bb0afaf5cd1c01025288af70d2935edff8ab90dd7c89d38cbf9094bb69 WatchSource:0}: Error finding container f3cb32bb0afaf5cd1c01025288af70d2935edff8ab90dd7c89d38cbf9094bb69: Status 404 returned error can't find the container with id f3cb32bb0afaf5cd1c01025288af70d2935edff8ab90dd7c89d38cbf9094bb69 Jun 11 13:18:52.255967 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.255917 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mj4f\" (UniqueName: \"kubernetes.io/projected/3760e24f-6a55-4805-ab9e-739ab04bbe86-kube-api-access-4mj4f\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:18:52.255967 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.255941 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3760e24f-6a55-4805-ab9e-739ab04bbe86-extensions-socket-volume\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:18:52.697413 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.697375 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" event={"ID":"d3ece720-508b-47fb-8434-15a19734ae7a","Type":"ContainerStarted","Data":"b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893"} Jun 11 13:18:52.697892 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.697530 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:18:52.697892 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.697579 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" event={"ID":"d3ece720-508b-47fb-8434-15a19734ae7a","Type":"ContainerStarted","Data":"f3cb32bb0afaf5cd1c01025288af70d2935edff8ab90dd7c89d38cbf9094bb69"} Jun 11 13:18:52.698722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.698700 2563 generic.go:356] "Generic (PLEG): container finished" podID="3760e24f-6a55-4805-ab9e-739ab04bbe86" containerID="5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef" exitCode=0 Jun 11 13:18:52.698863 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.698729 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" event={"ID":"3760e24f-6a55-4805-ab9e-739ab04bbe86","Type":"ContainerDied","Data":"5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef"} Jun 11 13:18:52.698863 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.698743 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" Jun 11 13:18:52.698863 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.698765 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd" event={"ID":"3760e24f-6a55-4805-ab9e-739ab04bbe86","Type":"ContainerDied","Data":"b339aeee293c349ee5858aef86ede33223f96878aad6594459bcde6a93e704f4"} Jun 11 13:18:52.698863 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.698790 2563 scope.go:117] "RemoveContainer" containerID="5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef" Jun 11 13:18:52.710318 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.710234 2563 scope.go:117] "RemoveContainer" containerID="5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef" Jun 11 13:18:52.710679 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:18:52.710650 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef\": container with ID starting with 5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef not found: ID does not exist" containerID="5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef" Jun 11 13:18:52.710780 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.710690 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef"} err="failed to get container status \"5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef\": rpc error: code = NotFound desc = could not find container \"5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef\": container with ID starting with 5c298a7e2785fa8a89ce1c00d6ccca21feee605c8859c7963ead7171129ae6ef not found: ID does not exist" Jun 11 13:18:52.721218 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.721178 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" podStartSLOduration=1.721165971 podStartE2EDuration="1.721165971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:18:52.720676055 +0000 UTC m=+495.963753888" watchObservedRunningTime="2026-06-11 13:18:52.721165971 +0000 UTC m=+495.964243803" Jun 11 13:18:52.739993 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.739970 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd"] Jun 11 13:18:52.746245 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:52.746224 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x4chd"] Jun 11 13:18:53.217750 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:18:53.217711 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3760e24f-6a55-4805-ab9e-739ab04bbe86" path="/var/lib/kubelet/pods/3760e24f-6a55-4805-ab9e-739ab04bbe86/volumes" Jun 11 13:19:03.707249 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:03.707223 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:19:10.170561 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.170527 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:19:10.171002 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.170928 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3760e24f-6a55-4805-ab9e-739ab04bbe86" containerName="manager" Jun 11 13:19:10.171002 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.170939 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3760e24f-6a55-4805-ab9e-739ab04bbe86" containerName="manager" Jun 11 13:19:10.171002 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.170999 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3760e24f-6a55-4805-ab9e-739ab04bbe86" containerName="manager" Jun 11 13:19:10.175677 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.175645 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:10.178270 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.178242 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 11 13:19:10.178402 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.178300 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jtv55\"" Jun 11 13:19:10.184713 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.184690 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:19:10.189278 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.189255 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9222d0d9-7353-48d3-866f-9905d73410f5-config-file\") pod \"limitador-limitador-54d6d988d5-s54rx\" (UID: \"9222d0d9-7353-48d3-866f-9905d73410f5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:10.189389 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.189335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfbf7\" (UniqueName: \"kubernetes.io/projected/9222d0d9-7353-48d3-866f-9905d73410f5-kube-api-access-jfbf7\") pod \"limitador-limitador-54d6d988d5-s54rx\" (UID: \"9222d0d9-7353-48d3-866f-9905d73410f5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:10.213394 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.213369 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:19:10.289944 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.289921 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfbf7\" (UniqueName: \"kubernetes.io/projected/9222d0d9-7353-48d3-866f-9905d73410f5-kube-api-access-jfbf7\") pod \"limitador-limitador-54d6d988d5-s54rx\" (UID: \"9222d0d9-7353-48d3-866f-9905d73410f5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:10.290069 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.289983 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9222d0d9-7353-48d3-866f-9905d73410f5-config-file\") pod \"limitador-limitador-54d6d988d5-s54rx\" (UID: \"9222d0d9-7353-48d3-866f-9905d73410f5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:10.290538 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.290520 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9222d0d9-7353-48d3-866f-9905d73410f5-config-file\") pod \"limitador-limitador-54d6d988d5-s54rx\" (UID: \"9222d0d9-7353-48d3-866f-9905d73410f5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:10.298259 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.298229 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfbf7\" (UniqueName: \"kubernetes.io/projected/9222d0d9-7353-48d3-866f-9905d73410f5-kube-api-access-jfbf7\") pod \"limitador-limitador-54d6d988d5-s54rx\" (UID: \"9222d0d9-7353-48d3-866f-9905d73410f5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:10.488248 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.488185 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:10.630693 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.630669 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:19:10.632890 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:19:10.632865 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9222d0d9_7353_48d3_866f_9905d73410f5.slice/crio-23eb93b92db0b5bc55dd00171457e719453d19c739def370fc7f4069d5d1e17b WatchSource:0}: Error finding container 23eb93b92db0b5bc55dd00171457e719453d19c739def370fc7f4069d5d1e17b: Status 404 returned error can't find the container with id 23eb93b92db0b5bc55dd00171457e719453d19c739def370fc7f4069d5d1e17b Jun 11 13:19:10.797678 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:10.797591 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" event={"ID":"9222d0d9-7353-48d3-866f-9905d73410f5","Type":"ContainerStarted","Data":"23eb93b92db0b5bc55dd00171457e719453d19c739def370fc7f4069d5d1e17b"} Jun 11 13:19:13.815849 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:13.815804 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" event={"ID":"9222d0d9-7353-48d3-866f-9905d73410f5","Type":"ContainerStarted","Data":"b3ec85f306553af513ce25d8bbd81c8333a0da7f8b41bd7e0df8779175cb3c73"} Jun 11 13:19:13.816278 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:13.815866 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:19:13.831284 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:13.831178 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" podStartSLOduration=1.3261319 podStartE2EDuration="3.831160369s" totalImagesPullingTime="2.505028469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:19:13.830742637 +0000 UTC m=+517.073820471" watchObservedRunningTime="2026-06-11 13:19:13.831160369 +0000 UTC m=+517.074238183" Jun 11 13:19:24.823194 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:19:24.823121 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-s54rx" Jun 11 13:20:44.993328 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:44.993293 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5ccc97c4b4-62lg6"] Jun 11 13:20:44.996516 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:44.996496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:44.999385 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:44.999361 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 11 13:20:44.999504 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:44.999374 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 11 13:20:44.999504 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:44.999412 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t4qnr\"" Jun 11 13:20:45.008502 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.008483 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5ccc97c4b4-62lg6"] Jun 11 13:20:45.160965 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.160911 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhr4p\" (UniqueName: \"kubernetes.io/projected/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-kube-api-access-xhr4p\") pod \"maas-controller-5ccc97c4b4-62lg6\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:45.161148 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.160990 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert\") pod \"maas-controller-5ccc97c4b4-62lg6\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:45.261755 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.261670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert\") pod \"maas-controller-5ccc97c4b4-62lg6\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:45.261896 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.261779 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhr4p\" (UniqueName: \"kubernetes.io/projected/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-kube-api-access-xhr4p\") pod \"maas-controller-5ccc97c4b4-62lg6\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:45.261896 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:20:45.261821 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 11 13:20:45.261896 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:20:45.261879 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert podName:b9ab5724-d099-4c3d-aa6a-bed90fb770a4 nodeName:}" failed. No retries permitted until 2026-06-11 13:20:45.76186232 +0000 UTC m=+609.004940132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert") pod "maas-controller-5ccc97c4b4-62lg6" (UID: "b9ab5724-d099-4c3d-aa6a-bed90fb770a4") : secret "maas-controller-webhook-cert" not found Jun 11 13:20:45.270454 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.270428 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhr4p\" (UniqueName: \"kubernetes.io/projected/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-kube-api-access-xhr4p\") pod \"maas-controller-5ccc97c4b4-62lg6\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:45.766994 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.766964 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert\") pod \"maas-controller-5ccc97c4b4-62lg6\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:45.769560 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.769526 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert\") pod \"maas-controller-5ccc97c4b4-62lg6\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:45.908177 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:45.908145 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:46.042977 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:46.042902 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5ccc97c4b4-62lg6"] Jun 11 13:20:46.045900 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:20:46.045871 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ab5724_d099_4c3d_aa6a_bed90fb770a4.slice/crio-ed9c56475d5f1f2f461d6491ee7a0b865e241a5cdc790fd9129a5243359c33f5 WatchSource:0}: Error finding container ed9c56475d5f1f2f461d6491ee7a0b865e241a5cdc790fd9129a5243359c33f5: Status 404 returned error can't find the container with id ed9c56475d5f1f2f461d6491ee7a0b865e241a5cdc790fd9129a5243359c33f5 Jun 11 13:20:46.281031 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:46.280997 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" event={"ID":"b9ab5724-d099-4c3d-aa6a-bed90fb770a4","Type":"ContainerStarted","Data":"ed9c56475d5f1f2f461d6491ee7a0b865e241a5cdc790fd9129a5243359c33f5"} Jun 11 13:20:49.298190 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:49.298156 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" event={"ID":"b9ab5724-d099-4c3d-aa6a-bed90fb770a4","Type":"ContainerStarted","Data":"37126d9fbec489bbf33cae222119602db293cc64745e4d41c637163829532bf5"} Jun 11 13:20:49.298588 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:49.298217 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:20:49.317097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:20:49.317050 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" podStartSLOduration=2.9613439870000002 podStartE2EDuration="5.317036884s" totalImagesPullingTime="2.355692897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:20:49.314734514 +0000 UTC m=+612.557812371" watchObservedRunningTime="2026-06-11 13:20:49.317036884 +0000 UTC m=+612.560114717" Jun 11 13:21:00.101251 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.101180 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5ccc97c4b4-62lg6"] Jun 11 13:21:00.101581 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.101393 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" podUID="b9ab5724-d099-4c3d-aa6a-bed90fb770a4" containerName="manager" containerID="cri-o://37126d9fbec489bbf33cae222119602db293cc64745e4d41c637163829532bf5" gracePeriod=30 Jun 11 13:21:00.112835 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.112813 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:21:00.354392 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.354323 2563 generic.go:356] "Generic (PLEG): container finished" podID="b9ab5724-d099-4c3d-aa6a-bed90fb770a4" containerID="37126d9fbec489bbf33cae222119602db293cc64745e4d41c637163829532bf5" exitCode=0 Jun 11 13:21:00.354392 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.354360 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" event={"ID":"b9ab5724-d099-4c3d-aa6a-bed90fb770a4","Type":"ContainerDied","Data":"37126d9fbec489bbf33cae222119602db293cc64745e4d41c637163829532bf5"} Jun 11 13:21:00.354554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.354399 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" event={"ID":"b9ab5724-d099-4c3d-aa6a-bed90fb770a4","Type":"ContainerDied","Data":"ed9c56475d5f1f2f461d6491ee7a0b865e241a5cdc790fd9129a5243359c33f5"} Jun 11 13:21:00.354554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.354415 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed9c56475d5f1f2f461d6491ee7a0b865e241a5cdc790fd9129a5243359c33f5" Jun 11 13:21:00.359794 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.359775 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:21:00.495604 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.495578 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert\") pod \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " Jun 11 13:21:00.495745 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.495611 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhr4p\" (UniqueName: \"kubernetes.io/projected/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-kube-api-access-xhr4p\") pod \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\" (UID: \"b9ab5724-d099-4c3d-aa6a-bed90fb770a4\") " Jun 11 13:21:00.497863 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.497836 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b9ab5724-d099-4c3d-aa6a-bed90fb770a4" (UID: "b9ab5724-d099-4c3d-aa6a-bed90fb770a4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:21:00.497959 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.497934 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-kube-api-access-xhr4p" (OuterVolumeSpecName: "kube-api-access-xhr4p") pod "b9ab5724-d099-4c3d-aa6a-bed90fb770a4" (UID: "b9ab5724-d099-4c3d-aa6a-bed90fb770a4"). InnerVolumeSpecName "kube-api-access-xhr4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:21:00.596574 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.596551 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-webhook-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:21:00.596574 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:00.596572 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhr4p\" (UniqueName: \"kubernetes.io/projected/b9ab5724-d099-4c3d-aa6a-bed90fb770a4-kube-api-access-xhr4p\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:21:01.359365 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:01.359328 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ccc97c4b4-62lg6" Jun 11 13:21:01.377554 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:01.377531 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5ccc97c4b4-62lg6"] Jun 11 13:21:01.383508 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:01.383473 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5ccc97c4b4-62lg6"] Jun 11 13:21:03.217669 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:03.217604 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9ab5724-d099-4c3d-aa6a-bed90fb770a4" path="/var/lib/kubelet/pods/b9ab5724-d099-4c3d-aa6a-bed90fb770a4/volumes" Jun 11 13:21:20.019067 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.019034 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-8swt6"] Jun 11 13:21:20.019529 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.019448 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ab5724-d099-4c3d-aa6a-bed90fb770a4" containerName="manager" Jun 11 13:21:20.019529 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.019459 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ab5724-d099-4c3d-aa6a-bed90fb770a4" containerName="manager" Jun 11 13:21:20.019529 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.019525 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ab5724-d099-4c3d-aa6a-bed90fb770a4" containerName="manager" Jun 11 13:21:20.022721 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.022700 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" Jun 11 13:21:20.025101 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.025076 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pz79q\"" Jun 11 13:21:20.025210 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.025082 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 11 13:21:20.034375 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.034338 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-8swt6"] Jun 11 13:21:20.060726 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.060692 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnshg\" (UniqueName: \"kubernetes.io/projected/f0147396-b9d6-446f-acc0-6c15687359e8-kube-api-access-mnshg\") pod \"payload-pre-processing-579785bc64-8swt6\" (UID: \"f0147396-b9d6-446f-acc0-6c15687359e8\") " pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" Jun 11 13:21:20.063869 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.063846 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:21:20.161203 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.161166 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnshg\" (UniqueName: \"kubernetes.io/projected/f0147396-b9d6-446f-acc0-6c15687359e8-kube-api-access-mnshg\") pod \"payload-pre-processing-579785bc64-8swt6\" (UID: \"f0147396-b9d6-446f-acc0-6c15687359e8\") " pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" Jun 11 13:21:20.169415 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.169393 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnshg\" (UniqueName: \"kubernetes.io/projected/f0147396-b9d6-446f-acc0-6c15687359e8-kube-api-access-mnshg\") pod \"payload-pre-processing-579785bc64-8swt6\" (UID: \"f0147396-b9d6-446f-acc0-6c15687359e8\") " pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" Jun 11 13:21:20.334096 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.334035 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" Jun 11 13:21:20.467284 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.467248 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-8swt6"] Jun 11 13:21:20.469261 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:21:20.469234 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0147396_b9d6_446f_acc0_6c15687359e8.slice/crio-c8ce196de0df8369b56e01581e730763ecc13308b2b5acac0946c48151459b26 WatchSource:0}: Error finding container c8ce196de0df8369b56e01581e730763ecc13308b2b5acac0946c48151459b26: Status 404 returned error can't find the container with id c8ce196de0df8369b56e01581e730763ecc13308b2b5acac0946c48151459b26 Jun 11 13:21:20.471107 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:20.471091 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 13:21:21.466473 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:21.466433 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" event={"ID":"f0147396-b9d6-446f-acc0-6c15687359e8","Type":"ContainerStarted","Data":"c8ce196de0df8369b56e01581e730763ecc13308b2b5acac0946c48151459b26"} Jun 11 13:21:23.477883 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:23.477848 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" event={"ID":"f0147396-b9d6-446f-acc0-6c15687359e8","Type":"ContainerStarted","Data":"3a2681539ec09ebda18d9300af64d52b2ebb1d79ca7f9846b1b75f8b04b1a051"} Jun 11 13:21:23.478268 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:23.478069 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" Jun 11 13:21:23.492200 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:23.492155 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" podStartSLOduration=2.305065485 podStartE2EDuration="4.492141785s" totalImagesPullingTime="2.1870763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:21:23.491755458 +0000 UTC m=+646.734833310" watchObservedRunningTime="2026-06-11 13:21:23.492141785 +0000 UTC m=+646.735219617" Jun 11 13:21:34.484732 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:21:34.484698 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-8swt6" Jun 11 13:24:33.464169 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:33.464129 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:24:37.260614 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:37.260576 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:24:47.744488 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.744455 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj"] Jun 11 13:24:47.748964 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.748939 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:47.752448 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.752425 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 11 13:24:47.752956 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.752427 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bvk66\"" Jun 11 13:24:47.753109 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.752437 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 11 13:24:47.753237 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.752550 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 11 13:24:47.755432 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.755410 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj"] Jun 11 13:24:47.913910 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.913879 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:47.914085 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.913946 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:47.914085 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.913983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:47.914085 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.914036 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:47.914085 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.914059 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdc9b\" (UniqueName: \"kubernetes.io/projected/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-kube-api-access-bdc9b\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:47.914243 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:47.914091 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015371 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015529 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015386 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015529 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015409 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015529 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015529 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015471 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdc9b\" (UniqueName: \"kubernetes.io/projected/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-kube-api-access-bdc9b\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015529 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015524 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015835 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015759 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015907 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015859 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.015978 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.015931 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.017938 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.017919 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.018115 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.018100 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.023418 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.023397 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdc9b\" (UniqueName: \"kubernetes.io/projected/bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4-kube-api-access-bdc9b\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj\" (UID: \"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.061946 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.061918 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:48.198167 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.198139 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj"] Jun 11 13:24:48.199975 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:24:48.199948 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc37b996_5bb2_46ae_a8b8_3cce98b8bdf4.slice/crio-31698304941cf09957c4953b62f731b848ccd4a339b570d5861637dcaee03a6b WatchSource:0}: Error finding container 31698304941cf09957c4953b62f731b848ccd4a339b570d5861637dcaee03a6b: Status 404 returned error can't find the container with id 31698304941cf09957c4953b62f731b848ccd4a339b570d5861637dcaee03a6b Jun 11 13:24:48.513744 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.513706 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" event={"ID":"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4","Type":"ContainerStarted","Data":"31698304941cf09957c4953b62f731b848ccd4a339b570d5861637dcaee03a6b"} Jun 11 13:24:48.660286 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:48.660250 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:24:53.541910 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:53.541873 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" event={"ID":"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4","Type":"ContainerStarted","Data":"a85039ef8af27cb28eaa8329c6f2f886438e8a982407bafcd4e929848f48b3d8"} Jun 11 13:24:54.548153 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:54.548116 2563 generic.go:356] "Generic (PLEG): container finished" podID="bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4" containerID="a85039ef8af27cb28eaa8329c6f2f886438e8a982407bafcd4e929848f48b3d8" exitCode=0 Jun 11 13:24:54.548513 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:54.548207 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" event={"ID":"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4","Type":"ContainerDied","Data":"a85039ef8af27cb28eaa8329c6f2f886438e8a982407bafcd4e929848f48b3d8"} Jun 11 13:24:56.562035 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:56.561958 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" event={"ID":"bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4","Type":"ContainerStarted","Data":"16c10f6f167f2e80903fc869fecf900827edfef6a881ca92bab851ca4ad86521"} Jun 11 13:24:56.583051 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:56.583008 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" podStartSLOduration=2.245885054 podStartE2EDuration="9.582993307s" totalImagesPullingTime="6.005853634s" totalInitContainerRuntime="1.331254619s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:24:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:24:56.580846546 +0000 UTC m=+859.823924425" watchObservedRunningTime="2026-06-11 13:24:56.582993307 +0000 UTC m=+859.826071139" Jun 11 13:24:58.062866 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:58.062818 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:58.063194 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:58.062936 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:58.075313 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:58.075293 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:24:58.583064 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:24:58.583037 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj" Jun 11 13:25:02.770950 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:02.770916 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:25:07.164115 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.164083 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd"] Jun 11 13:25:07.168072 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.168054 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.170692 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.170666 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 11 13:25:07.175638 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.175420 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd"] Jun 11 13:25:07.187543 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.187518 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.187669 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.187556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.187669 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.187574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tpjp\" (UniqueName: \"kubernetes.io/projected/e4257962-0645-4ea9-af88-fb0c6e4d693d-kube-api-access-4tpjp\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.187669 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.187593 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.187789 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.187690 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4257962-0645-4ea9-af88-fb0c6e4d693d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.187789 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.187724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.288926 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.288902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.288926 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.288937 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.289128 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.288954 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tpjp\" (UniqueName: \"kubernetes.io/projected/e4257962-0645-4ea9-af88-fb0c6e4d693d-kube-api-access-4tpjp\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.289128 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.288970 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.289128 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.289001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4257962-0645-4ea9-af88-fb0c6e4d693d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.289128 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.289026 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.289370 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.289330 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.289370 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.289344 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.289489 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.289460 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.291677 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.291651 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4257962-0645-4ea9-af88-fb0c6e4d693d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.291797 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.291777 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4257962-0645-4ea9-af88-fb0c6e4d693d-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.297373 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.297348 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tpjp\" (UniqueName: \"kubernetes.io/projected/e4257962-0645-4ea9-af88-fb0c6e4d693d-kube-api-access-4tpjp\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd\" (UID: \"e4257962-0645-4ea9-af88-fb0c6e4d693d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.480286 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.480218 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:07.618970 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:07.618942 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd"] Jun 11 13:25:07.620500 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:25:07.620472 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4257962_0645_4ea9_af88_fb0c6e4d693d.slice/crio-f7f79ec8c77ea858a701d49b41547ed6372d9157adca3d7870ee06e26399b46e WatchSource:0}: Error finding container f7f79ec8c77ea858a701d49b41547ed6372d9157adca3d7870ee06e26399b46e: Status 404 returned error can't find the container with id f7f79ec8c77ea858a701d49b41547ed6372d9157adca3d7870ee06e26399b46e Jun 11 13:25:08.265897 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:08.265866 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:25:08.624898 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:08.624822 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" event={"ID":"e4257962-0645-4ea9-af88-fb0c6e4d693d","Type":"ContainerStarted","Data":"5f106963f2a59365d36a30ed7785ba430a23c77fb2d2d80376d232db6391142c"} Jun 11 13:25:08.624898 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:08.624877 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" event={"ID":"e4257962-0645-4ea9-af88-fb0c6e4d693d","Type":"ContainerStarted","Data":"f7f79ec8c77ea858a701d49b41547ed6372d9157adca3d7870ee06e26399b46e"} Jun 11 13:25:09.632144 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:09.632107 2563 generic.go:356] "Generic (PLEG): container finished" podID="e4257962-0645-4ea9-af88-fb0c6e4d693d" containerID="5f106963f2a59365d36a30ed7785ba430a23c77fb2d2d80376d232db6391142c" exitCode=0 Jun 11 13:25:09.632537 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:09.632191 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" event={"ID":"e4257962-0645-4ea9-af88-fb0c6e4d693d","Type":"ContainerDied","Data":"5f106963f2a59365d36a30ed7785ba430a23c77fb2d2d80376d232db6391142c"} Jun 11 13:25:10.638799 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:10.638764 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" event={"ID":"e4257962-0645-4ea9-af88-fb0c6e4d693d","Type":"ContainerStarted","Data":"b9d49c038b8e7ad7036f428cf0c057306806b3794bf02cad6908e74c0ca9a13c"} Jun 11 13:25:10.658098 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:10.658049 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" podStartSLOduration=2.044364556 podStartE2EDuration="3.658033821s" totalImagesPullingTime="252.468312ms" totalInitContainerRuntime="1.361200953s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:25:10.656047418 +0000 UTC m=+873.899125251" watchObservedRunningTime="2026-06-11 13:25:10.658033821 +0000 UTC m=+873.901111655" Jun 11 13:25:17.481193 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:17.481162 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:17.481565 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:17.481214 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:17.493889 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:17.493868 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:17.695567 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:17.695542 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd" Jun 11 13:25:20.075030 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:25:20.074999 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:26:58.472107 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:26:58.472026 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:27:07.867057 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:27:07.867019 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:27:17.767788 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:27:17.767752 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:27:27.658750 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:27:27.658703 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:27:37.063856 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:27:37.063820 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:27:37.271078 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:27:37.271046 2563 scope.go:117] "RemoveContainer" containerID="37126d9fbec489bbf33cae222119602db293cc64745e4d41c637163829532bf5" Jun 11 13:27:45.969653 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:27:45.969602 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:28:19.867401 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:28:19.867365 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:28:23.566097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:28:23.566058 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:28:59.571023 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:28:59.570983 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:29:15.999466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:15.999431 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:29:31.869266 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.869233 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cbfbdcc5d-wkh8d"] Jun 11 13:29:31.872733 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.872708 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:31.885721 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.885693 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cbfbdcc5d-wkh8d"] Jun 11 13:29:31.958662 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.958607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-oauth-serving-cert\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:31.958803 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.958674 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-service-ca\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:31.958803 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.958783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlhgk\" (UniqueName: \"kubernetes.io/projected/3c537f2d-294c-48fc-a921-1f0ba40e18d4-kube-api-access-xlhgk\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:31.958928 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.958845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-config\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:31.958928 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.958882 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-trusted-ca-bundle\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:31.959032 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.958957 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-serving-cert\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:31.959032 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:31.959001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-oauth-config\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.060301 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.060275 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-serving-cert\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.060450 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.060316 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-oauth-config\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.060450 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.060346 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-oauth-serving-cert\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.060450 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.060372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-service-ca\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.060450 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.060414 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhgk\" (UniqueName: \"kubernetes.io/projected/3c537f2d-294c-48fc-a921-1f0ba40e18d4-kube-api-access-xlhgk\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.060697 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.060457 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-config\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.060697 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.060501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-trusted-ca-bundle\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.061272 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.061249 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-config\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.061377 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.061286 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-oauth-serving-cert\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.061423 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.061377 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-service-ca\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.061423 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.061381 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c537f2d-294c-48fc-a921-1f0ba40e18d4-trusted-ca-bundle\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.063010 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.062980 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-serving-cert\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.063107 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.063035 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c537f2d-294c-48fc-a921-1f0ba40e18d4-console-oauth-config\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.068268 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.068248 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlhgk\" (UniqueName: \"kubernetes.io/projected/3c537f2d-294c-48fc-a921-1f0ba40e18d4-kube-api-access-xlhgk\") pod \"console-6cbfbdcc5d-wkh8d\" (UID: \"3c537f2d-294c-48fc-a921-1f0ba40e18d4\") " pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.182942 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.182916 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:32.317860 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:32.317829 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cbfbdcc5d-wkh8d"] Jun 11 13:29:32.319973 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:29:32.319948 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c537f2d_294c_48fc_a921_1f0ba40e18d4.slice/crio-9260c9df8fd87b231612f17d6209966a80af8c5bf0078a14b290a696229cf929 WatchSource:0}: Error finding container 9260c9df8fd87b231612f17d6209966a80af8c5bf0078a14b290a696229cf929: Status 404 returned error can't find the container with id 9260c9df8fd87b231612f17d6209966a80af8c5bf0078a14b290a696229cf929 Jun 11 13:29:33.035466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:33.035423 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cbfbdcc5d-wkh8d" event={"ID":"3c537f2d-294c-48fc-a921-1f0ba40e18d4","Type":"ContainerStarted","Data":"baf26f7bfa3f2852f5973e507e5b828692f325032a175f72345d19a473bf09f7"} Jun 11 13:29:33.035934 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:33.035533 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cbfbdcc5d-wkh8d" event={"ID":"3c537f2d-294c-48fc-a921-1f0ba40e18d4","Type":"ContainerStarted","Data":"9260c9df8fd87b231612f17d6209966a80af8c5bf0078a14b290a696229cf929"} Jun 11 13:29:33.054496 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:33.054449 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cbfbdcc5d-wkh8d" podStartSLOduration=2.054433957 podStartE2EDuration="2.054433957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:29:33.052832372 +0000 UTC m=+1136.295910203" watchObservedRunningTime="2026-06-11 13:29:33.054433957 +0000 UTC m=+1136.297511789" Jun 11 13:29:42.183467 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:42.183429 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:42.183973 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:42.183500 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:42.188302 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:42.188277 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:43.091348 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:43.091321 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cbfbdcc5d-wkh8d" Jun 11 13:29:43.138205 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:43.138176 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb5954bbc-s7x9v"] Jun 11 13:29:54.270080 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:29:54.270050 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:30:08.166085 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.166031 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5cb5954bbc-s7x9v" podUID="11daf53c-114c-4256-9ef0-649c017d7726" containerName="console" containerID="cri-o://d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490" gracePeriod=15 Jun 11 13:30:08.426341 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.426292 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:30:08.478515 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.478487 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-service-ca\") pod \"11daf53c-114c-4256-9ef0-649c017d7726\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " Jun 11 13:30:08.478515 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.478518 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-console-config\") pod \"11daf53c-114c-4256-9ef0-649c017d7726\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " Jun 11 13:30:08.478736 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.478552 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-trusted-ca-bundle\") pod \"11daf53c-114c-4256-9ef0-649c017d7726\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " Jun 11 13:30:08.478736 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.478686 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-oauth-serving-cert\") pod \"11daf53c-114c-4256-9ef0-649c017d7726\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " Jun 11 13:30:08.478736 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.478724 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-serving-cert\") pod \"11daf53c-114c-4256-9ef0-649c017d7726\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " Jun 11 13:30:08.478899 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.478806 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wfxq\" (UniqueName: \"kubernetes.io/projected/11daf53c-114c-4256-9ef0-649c017d7726-kube-api-access-2wfxq\") pod \"11daf53c-114c-4256-9ef0-649c017d7726\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " Jun 11 13:30:08.478899 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.478835 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-oauth-config\") pod \"11daf53c-114c-4256-9ef0-649c017d7726\" (UID: \"11daf53c-114c-4256-9ef0-649c017d7726\") " Jun 11 13:30:08.479048 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.479018 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-console-config" (OuterVolumeSpecName: "console-config") pod "11daf53c-114c-4256-9ef0-649c017d7726" (UID: "11daf53c-114c-4256-9ef0-649c017d7726"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:30:08.479048 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.479033 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-service-ca" (OuterVolumeSpecName: "service-ca") pod "11daf53c-114c-4256-9ef0-649c017d7726" (UID: "11daf53c-114c-4256-9ef0-649c017d7726"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:30:08.479152 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.479051 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "11daf53c-114c-4256-9ef0-649c017d7726" (UID: "11daf53c-114c-4256-9ef0-649c017d7726"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:30:08.479202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.479172 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "11daf53c-114c-4256-9ef0-649c017d7726" (UID: "11daf53c-114c-4256-9ef0-649c017d7726"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 13:30:08.479202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.479192 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-service-ca\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:30:08.479302 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.479208 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-console-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:30:08.479302 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.479217 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-trusted-ca-bundle\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:30:08.480919 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.480893 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "11daf53c-114c-4256-9ef0-649c017d7726" (UID: "11daf53c-114c-4256-9ef0-649c017d7726"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:30:08.481013 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.480986 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11daf53c-114c-4256-9ef0-649c017d7726-kube-api-access-2wfxq" (OuterVolumeSpecName: "kube-api-access-2wfxq") pod "11daf53c-114c-4256-9ef0-649c017d7726" (UID: "11daf53c-114c-4256-9ef0-649c017d7726"). InnerVolumeSpecName "kube-api-access-2wfxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:30:08.481056 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.481038 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "11daf53c-114c-4256-9ef0-649c017d7726" (UID: "11daf53c-114c-4256-9ef0-649c017d7726"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 13:30:08.580172 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.580147 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11daf53c-114c-4256-9ef0-649c017d7726-oauth-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:30:08.580172 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.580167 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-serving-cert\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:30:08.580172 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.580175 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wfxq\" (UniqueName: \"kubernetes.io/projected/11daf53c-114c-4256-9ef0-649c017d7726-kube-api-access-2wfxq\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:30:08.580386 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:08.580183 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11daf53c-114c-4256-9ef0-649c017d7726-console-oauth-config\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:30:09.234298 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.234273 2563 generic.go:356] "Generic (PLEG): container finished" podID="11daf53c-114c-4256-9ef0-649c017d7726" containerID="d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490" exitCode=0 Jun 11 13:30:09.234612 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.234316 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb5954bbc-s7x9v" Jun 11 13:30:09.234612 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.234351 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb5954bbc-s7x9v" event={"ID":"11daf53c-114c-4256-9ef0-649c017d7726","Type":"ContainerDied","Data":"d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490"} Jun 11 13:30:09.234612 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.234396 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb5954bbc-s7x9v" event={"ID":"11daf53c-114c-4256-9ef0-649c017d7726","Type":"ContainerDied","Data":"b861eb3a0fd3d868ed58b4bc7e4632e599670a3384a9ab7099d99ab78c64de1a"} Jun 11 13:30:09.234612 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.234418 2563 scope.go:117] "RemoveContainer" containerID="d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490" Jun 11 13:30:09.243346 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.243327 2563 scope.go:117] "RemoveContainer" containerID="d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490" Jun 11 13:30:09.243592 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:30:09.243571 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490\": container with ID starting with d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490 not found: ID does not exist" containerID="d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490" Jun 11 13:30:09.243701 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.243598 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490"} err="failed to get container status \"d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490\": rpc error: code = NotFound desc = could not find container \"d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490\": container with ID starting with d00e4d7d99fb7fedf7a2a3815e1134a2ba3982a93aec1672867c09fa4841f490 not found: ID does not exist" Jun 11 13:30:09.251392 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.251371 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb5954bbc-s7x9v"] Jun 11 13:30:09.255407 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:09.255387 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cb5954bbc-s7x9v"] Jun 11 13:30:11.065260 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:11.065223 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:30:11.218073 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:11.218038 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11daf53c-114c-4256-9ef0-649c017d7726" path="/var/lib/kubelet/pods/11daf53c-114c-4256-9ef0-649c017d7726/volumes" Jun 11 13:30:25.766412 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:25.766377 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:30:40.677710 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:40.677668 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:30:44.664333 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:30:44.664299 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:31:16.160301 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:31:16.160266 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:31:20.275643 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:31:20.274220 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:31:41.460394 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:31:41.460313 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:31:50.469129 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:31:50.469093 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:32:07.065896 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:32:07.065863 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:32:15.662154 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:32:15.662113 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:32:33.131568 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:32:33.131533 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:32:41.558367 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:32:41.558333 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:33:13.971988 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:33:13.971908 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:33:23.254261 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:33:23.254227 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:33:30.669289 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:33:30.669250 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:33:39.067432 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:33:39.067397 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:33:47.660682 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:33:47.660645 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:33:56.266355 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:33:56.266322 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:34:00.106596 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:34:00.106563 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:34:13.068184 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:34:13.068145 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:34:25.170202 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:34:25.170118 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:35:13.260218 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:35:13.260176 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:35:19.767138 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:35:19.767100 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:35:38.760786 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:35:38.760754 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:35:43.660386 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:35:43.660356 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:35:47.661050 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:35:47.661014 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:35:57.372462 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:35:57.372387 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:36:05.063920 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:36:05.063882 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:36:15.061811 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:36:15.061775 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:36:22.561546 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:36:22.561514 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:36:32.664951 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:36:32.664905 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:36:40.065023 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:36:40.064990 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:36:48.170299 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:36:48.170258 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:36:56.566068 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:36:56.566037 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:37:58.176379 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.176290 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7"] Jun 11 13:37:58.176914 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.176532 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" podUID="d3ece720-508b-47fb-8434-15a19734ae7a" containerName="manager" containerID="cri-o://b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893" gracePeriod=10 Jun 11 13:37:58.437301 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.437243 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:37:58.583457 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.583430 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzpkm\" (UniqueName: \"kubernetes.io/projected/d3ece720-508b-47fb-8434-15a19734ae7a-kube-api-access-pzpkm\") pod \"d3ece720-508b-47fb-8434-15a19734ae7a\" (UID: \"d3ece720-508b-47fb-8434-15a19734ae7a\") " Jun 11 13:37:58.583601 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.583487 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ece720-508b-47fb-8434-15a19734ae7a-extensions-socket-volume\") pod \"d3ece720-508b-47fb-8434-15a19734ae7a\" (UID: \"d3ece720-508b-47fb-8434-15a19734ae7a\") " Jun 11 13:37:58.583867 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.583848 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ece720-508b-47fb-8434-15a19734ae7a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d3ece720-508b-47fb-8434-15a19734ae7a" (UID: "d3ece720-508b-47fb-8434-15a19734ae7a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 13:37:58.585574 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.585549 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ece720-508b-47fb-8434-15a19734ae7a-kube-api-access-pzpkm" (OuterVolumeSpecName: "kube-api-access-pzpkm") pod "d3ece720-508b-47fb-8434-15a19734ae7a" (UID: "d3ece720-508b-47fb-8434-15a19734ae7a"). InnerVolumeSpecName "kube-api-access-pzpkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 13:37:58.684726 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.684701 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzpkm\" (UniqueName: \"kubernetes.io/projected/d3ece720-508b-47fb-8434-15a19734ae7a-kube-api-access-pzpkm\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:37:58.684726 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.684722 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ece720-508b-47fb-8434-15a19734ae7a-extensions-socket-volume\") on node \"ip-10-0-141-252.ec2.internal\" DevicePath \"\"" Jun 11 13:37:58.696556 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.696487 2563 generic.go:356] "Generic (PLEG): container finished" podID="d3ece720-508b-47fb-8434-15a19734ae7a" containerID="b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893" exitCode=0 Jun 11 13:37:58.696556 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.696541 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" Jun 11 13:37:58.696742 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.696571 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" event={"ID":"d3ece720-508b-47fb-8434-15a19734ae7a","Type":"ContainerDied","Data":"b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893"} Jun 11 13:37:58.696742 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.696643 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7" event={"ID":"d3ece720-508b-47fb-8434-15a19734ae7a","Type":"ContainerDied","Data":"f3cb32bb0afaf5cd1c01025288af70d2935edff8ab90dd7c89d38cbf9094bb69"} Jun 11 13:37:58.696742 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.696677 2563 scope.go:117] "RemoveContainer" containerID="b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893" Jun 11 13:37:58.705956 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.705937 2563 scope.go:117] "RemoveContainer" containerID="b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893" Jun 11 13:37:58.706219 ip-10-0-141-252 kubenswrapper[2563]: E0611 13:37:58.706200 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893\": container with ID starting with b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893 not found: ID does not exist" containerID="b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893" Jun 11 13:37:58.706272 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.706227 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893"} err="failed to get container status \"b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893\": rpc error: code = NotFound desc = could not find container \"b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893\": container with ID starting with b98b0bcc610d0df95e7e93dc5bca0e49597ef7860bab3e8fdb6aa6846cd2c893 not found: ID does not exist" Jun 11 13:37:58.718553 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.718532 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7"] Jun 11 13:37:58.722466 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:58.722441 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8z5k7"] Jun 11 13:37:59.218809 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:37:59.218781 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3ece720-508b-47fb-8434-15a19734ae7a" path="/var/lib/kubelet/pods/d3ece720-508b-47fb-8434-15a19734ae7a/volumes" Jun 11 13:39:04.451044 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.450967 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5"] Jun 11 13:39:04.451414 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.451360 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ece720-508b-47fb-8434-15a19734ae7a" containerName="manager" Jun 11 13:39:04.451414 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.451372 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ece720-508b-47fb-8434-15a19734ae7a" containerName="manager" Jun 11 13:39:04.451414 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.451391 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11daf53c-114c-4256-9ef0-649c017d7726" containerName="console" Jun 11 13:39:04.451414 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.451397 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="11daf53c-114c-4256-9ef0-649c017d7726" containerName="console" Jun 11 13:39:04.451555 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.451469 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="11daf53c-114c-4256-9ef0-649c017d7726" containerName="console" Jun 11 13:39:04.451555 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.451477 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ece720-508b-47fb-8434-15a19734ae7a" containerName="manager" Jun 11 13:39:04.454496 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.454480 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:04.457685 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.457669 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nbdpr\"" Jun 11 13:39:04.501265 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.501241 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5"] Jun 11 13:39:04.514691 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.514668 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bzwv5\" (UID: \"e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:04.514822 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.514732 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkrjg\" (UniqueName: \"kubernetes.io/projected/e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b-kube-api-access-vkrjg\") pod \"kuadrant-operator-controller-manager-86878c769f-bzwv5\" (UID: \"e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:04.615718 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.615687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bzwv5\" (UID: \"e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:04.615871 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.615749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkrjg\" (UniqueName: \"kubernetes.io/projected/e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b-kube-api-access-vkrjg\") pod \"kuadrant-operator-controller-manager-86878c769f-bzwv5\" (UID: \"e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:04.616074 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.616054 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bzwv5\" (UID: \"e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:04.633185 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.633153 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkrjg\" (UniqueName: \"kubernetes.io/projected/e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b-kube-api-access-vkrjg\") pod \"kuadrant-operator-controller-manager-86878c769f-bzwv5\" (UID: \"e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:04.764785 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.764712 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:04.904031 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:04.904005 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5"] Jun 11 13:39:04.906647 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:39:04.906598 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode732a0fb_91fc_4114_9c54_1ed0d5e5ae4b.slice/crio-21bb612eb61eb0b974b84e372de6dec1415d0651e9247fa476699e8e4625829a WatchSource:0}: Error finding container 21bb612eb61eb0b974b84e372de6dec1415d0651e9247fa476699e8e4625829a: Status 404 returned error can't find the container with id 21bb612eb61eb0b974b84e372de6dec1415d0651e9247fa476699e8e4625829a Jun 11 13:39:05.051930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:05.051851 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" event={"ID":"e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b","Type":"ContainerStarted","Data":"d04c15c4e18b1cb08129696223852608f7480c5369579088fd1aefaa836e4d9a"} Jun 11 13:39:05.051930 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:05.051915 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" event={"ID":"e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b","Type":"ContainerStarted","Data":"21bb612eb61eb0b974b84e372de6dec1415d0651e9247fa476699e8e4625829a"} Jun 11 13:39:05.052137 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:05.051972 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:05.071111 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:05.071070 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" podStartSLOduration=1.071054953 podStartE2EDuration="1.071054953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 13:39:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:39:05.069369191 +0000 UTC m=+1708.312447035" watchObservedRunningTime="2026-06-11 13:39:05.071054953 +0000 UTC m=+1708.314132839" Jun 11 13:39:16.060109 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:16.060078 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bzwv5" Jun 11 13:39:26.262679 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:26.262638 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:39:30.783553 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:30.783515 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:39:56.366077 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:39:56.366044 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:40:01.364949 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:40:01.364910 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:40:08.465666 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:40:08.465613 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:40:19.265504 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:40:19.265468 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:40:27.565132 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:40:27.565049 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:40:38.468303 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:40:38.468267 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:40:47.176128 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:40:47.176085 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:40:57.360483 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:40:57.360449 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:41:06.460848 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:41:06.460810 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:41:16.859722 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:41:16.859684 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:41:26.066801 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:41:26.066768 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:41:30.663995 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:41:30.663957 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:41:58.860405 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:41:58.860323 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:42:41.665519 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:42:41.665484 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:42:50.462282 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:42:50.462243 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:42:59.073493 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:42:59.073460 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:43:07.670866 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:43:07.670826 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:43:16.063267 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:43:16.063237 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:43:26.659849 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:43:26.659768 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:43:35.658825 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:43:35.658788 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:43:40.769543 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:43:40.769512 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:43:47.867597 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:43:47.867557 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:43:56.556461 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:43:56.556430 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:44:04.368233 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:44:04.368189 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:44:14.760409 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:44:14.760365 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:44:32.769915 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:44:32.769879 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:44:41.764316 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:44:41.764274 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:44:49.961540 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:44:49.961503 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:44:58.765444 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:44:58.765363 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:45:15.374330 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:45:15.374296 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:45:23.357967 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:45:23.357931 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:45:32.612741 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:45:32.612705 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:45:40.659792 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:45:40.659756 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:45:49.768773 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:45:49.768735 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:45:58.166304 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:45:58.166269 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:46:07.265371 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:46:07.265336 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:46:18.058676 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:46:18.058641 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:46:27.362616 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:46:27.362531 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:46:38.257782 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:46:38.257750 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:46:47.573215 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:46:47.573180 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:46:52.477670 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:46:52.477616 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:46:59.863710 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:46:59.863667 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:47:04.858092 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:47:04.858050 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:47:19.661464 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:47:19.661426 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:47:28.370536 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:47:28.370499 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s54rx"] Jun 11 13:48:15.337406 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:15.337375 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-68c659989f-ghq4z_a9326b61-159d-4b6e-9e32-9a015a1e2182/nginx/0.log" Jun 11 13:48:15.598413 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:15.598287 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-tdlg2_3638921d-7768-489d-988f-87e9a06e5b64/manager/0.log" Jun 11 13:48:15.716097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:15.716070 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-z2hqn_82a9297a-b0c2-43f5-bab1-753963757bd4/manager/0.log" Jun 11 13:48:16.301910 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:16.301884 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cdf5bf854-7nccn_95965310-296e-4f56-b9e0-72f7690f2a3f/copy-manifests/0.log" Jun 11 13:48:16.308834 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:16.308809 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cdf5bf854-7nccn_95965310-296e-4f56-b9e0-72f7690f2a3f/manager/0.log" Jun 11 13:48:17.922039 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:17.922012 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-c8lht_3772cbd4-6e28-411d-a432-8e2597e59863/manager/0.log" Jun 11 13:48:18.256315 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:18.256222 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bzwv5_e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b/manager/0.log" Jun 11 13:48:18.362296 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:18.362263 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-s54rx_9222d0d9-7353-48d3-866f-9905d73410f5/limitador/0.log" Jun 11 13:48:18.482830 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:18.482799 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-68bp8_8e928376-fb70-442a-9d86-e3d86555d2ad/manager/0.log" Jun 11 13:48:18.931641 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:18.931593 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-z57z9_691275c6-2abb-47a8-ab09-dccc20061027/discovery/0.log" Jun 11 13:48:19.037725 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:19.037699 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f55b6cf49-4btt9_9106ac49-59aa-4d34-8443-10d1d72aa60f/kube-auth-proxy/0.log" Jun 11 13:48:19.386769 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:19.386686 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8swt6_f0147396-b9d6-446f-acc0-6c15687359e8/payload-pre-processing/0.log" Jun 11 13:48:20.184525 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:20.184486 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj_bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4/main/0.log" Jun 11 13:48:20.190653 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:20.190614 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-9fsbj_bc37b996-5bb2-46ae-a8b8-3cce98b8bdf4/storage-initializer/0.log" Jun 11 13:48:20.415403 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:20.415376 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd_e4257962-0645-4ea9-af88-fb0c6e4d693d/main/0.log" Jun 11 13:48:20.421759 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:20.421737 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-wbdnd_e4257962-0645-4ea9-af88-fb0c6e4d693d/storage-initializer/0.log" Jun 11 13:48:27.551832 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:27.551777 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6xs65_fddd497e-da5a-4f4f-adcf-7041387c2d92/global-pull-secret-syncer/0.log" Jun 11 13:48:27.718149 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:27.718123 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p5hp9_7c5689cf-8303-455c-b659-5d3f9714d416/konnectivity-agent/0.log" Jun 11 13:48:27.794097 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:27.794072 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-252.ec2.internal_5b732421dcc2171bb17036784d182879/haproxy/0.log" Jun 11 13:48:32.216889 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:32.216862 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-ks74c_bfc8c0b6-0fa9-414a-b70c-fe7fa762bcc9/sail-operator/0.log" Jun 11 13:48:32.737381 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:32.737350 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-c8lht_3772cbd4-6e28-411d-a432-8e2597e59863/manager/0.log" Jun 11 13:48:32.883827 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:32.883802 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bzwv5_e732a0fb-91fc-4114-9c54-1ed0d5e5ae4b/manager/0.log" Jun 11 13:48:32.913235 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:32.913212 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-s54rx_9222d0d9-7353-48d3-866f-9905d73410f5/limitador/0.log" Jun 11 13:48:33.004636 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:33.004557 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-68bp8_8e928376-fb70-442a-9d86-e3d86555d2ad/manager/0.log" Jun 11 13:48:34.647569 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.647537 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_199ab944-4f6e-4f60-a67b-a6de7a59025b/alertmanager/0.log" Jun 11 13:48:34.669537 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.669514 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_199ab944-4f6e-4f60-a67b-a6de7a59025b/config-reloader/0.log" Jun 11 13:48:34.695910 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.695883 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_199ab944-4f6e-4f60-a67b-a6de7a59025b/kube-rbac-proxy-web/0.log" Jun 11 13:48:34.716828 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.716805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_199ab944-4f6e-4f60-a67b-a6de7a59025b/kube-rbac-proxy/0.log" Jun 11 13:48:34.743488 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.743469 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_199ab944-4f6e-4f60-a67b-a6de7a59025b/kube-rbac-proxy-metric/0.log" Jun 11 13:48:34.758474 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.758454 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_199ab944-4f6e-4f60-a67b-a6de7a59025b/prom-label-proxy/0.log" Jun 11 13:48:34.782114 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.782091 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_199ab944-4f6e-4f60-a67b-a6de7a59025b/init-config-reloader/0.log" Jun 11 13:48:34.924379 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.924355 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-859cd447cd-wh2xg_d0314b1e-f6a4-4b8e-be3d-b8a6da1dfd73/metrics-server/0.log" Jun 11 13:48:34.950576 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:34.950549 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-c9vwr_6b06cf1f-8154-42c0-99d1-2441a389f92a/monitoring-plugin/0.log" Jun 11 13:48:35.152553 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.152517 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l6q47_5780a6e1-f323-4d22-bdb1-e937222424e7/node-exporter/0.log" Jun 11 13:48:35.173517 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.173491 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l6q47_5780a6e1-f323-4d22-bdb1-e937222424e7/kube-rbac-proxy/0.log" Jun 11 13:48:35.195341 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.195268 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l6q47_5780a6e1-f323-4d22-bdb1-e937222424e7/init-textfile/0.log" Jun 11 13:48:35.225504 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.225477 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-6ksvb_de760bf0-c4f8-4281-980c-3abca5f7d0a7/kube-rbac-proxy-main/0.log" Jun 11 13:48:35.254507 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.254484 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-6ksvb_de760bf0-c4f8-4281-980c-3abca5f7d0a7/kube-rbac-proxy-self/0.log" Jun 11 13:48:35.275333 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.275312 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-6ksvb_de760bf0-c4f8-4281-980c-3abca5f7d0a7/openshift-state-metrics/0.log" Jun 11 13:48:35.508238 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.508154 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-6jxfx_5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5/prometheus-operator/0.log" Jun 11 13:48:35.529082 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.529056 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-6jxfx_5bfe60de-e57a-4ea4-b4d4-bcc3c8ae46d5/kube-rbac-proxy/0.log" Jun 11 13:48:35.559399 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.559380 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-5zhvr_47eafdac-30ee-4c3c-9776-20e525434f7c/prometheus-operator-admission-webhook/0.log" Jun 11 13:48:35.592153 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.592131 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69cd66d5bf-mxsns_41ad60cd-584b-49c1-b7be-83b7abcdebf8/telemeter-client/0.log" Jun 11 13:48:35.625720 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.625702 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69cd66d5bf-mxsns_41ad60cd-584b-49c1-b7be-83b7abcdebf8/reload/0.log" Jun 11 13:48:35.647252 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.647230 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69cd66d5bf-mxsns_41ad60cd-584b-49c1-b7be-83b7abcdebf8/kube-rbac-proxy/0.log" Jun 11 13:48:35.681434 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.681417 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-766c45c974-tqzz4_a96a724d-26c4-469a-bdb2-7c235d235ee4/thanos-query/0.log" Jun 11 13:48:35.704526 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.704505 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-766c45c974-tqzz4_a96a724d-26c4-469a-bdb2-7c235d235ee4/kube-rbac-proxy-web/0.log" Jun 11 13:48:35.729264 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.729245 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-766c45c974-tqzz4_a96a724d-26c4-469a-bdb2-7c235d235ee4/kube-rbac-proxy/0.log" Jun 11 13:48:35.757324 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.757308 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-766c45c974-tqzz4_a96a724d-26c4-469a-bdb2-7c235d235ee4/prom-label-proxy/0.log" Jun 11 13:48:35.779220 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.779150 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-766c45c974-tqzz4_a96a724d-26c4-469a-bdb2-7c235d235ee4/kube-rbac-proxy-rules/0.log" Jun 11 13:48:35.800280 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:35.800260 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-766c45c974-tqzz4_a96a724d-26c4-469a-bdb2-7c235d235ee4/kube-rbac-proxy-metrics/0.log" Jun 11 13:48:38.396246 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:38.396218 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6cbfbdcc5d-wkh8d_3c537f2d-294c-48fc-a921-1f0ba40e18d4/console/0.log" Jun 11 13:48:38.433725 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:38.433695 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-stw6h_032c44d5-58a5-4dd2-8c67-97e4828bfbcb/download-server/0.log" Jun 11 13:48:39.757233 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.757192 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4"] Jun 11 13:48:39.761002 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.760983 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.763937 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.763910 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5gk42\"/\"openshift-service-ca.crt\"" Jun 11 13:48:39.764076 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.764033 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5gk42\"/\"default-dockercfg-jfczr\"" Jun 11 13:48:39.765122 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.765100 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5gk42\"/\"kube-root-ca.crt\"" Jun 11 13:48:39.769878 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.769853 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4"] Jun 11 13:48:39.827964 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.827937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-sys\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.828074 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.827998 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-lib-modules\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.828074 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.828049 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-podres\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.828074 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.828065 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmjj2\" (UniqueName: \"kubernetes.io/projected/891604c5-ed92-44e4-8287-e49e84460dd8-kube-api-access-mmjj2\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.828183 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.828086 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-proc\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929245 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929218 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-sys\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929342 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-lib-modules\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929342 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-podres\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929457 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929345 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-sys\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929457 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929352 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmjj2\" (UniqueName: \"kubernetes.io/projected/891604c5-ed92-44e4-8287-e49e84460dd8-kube-api-access-mmjj2\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929457 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929394 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-proc\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929568 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929461 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-podres\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929568 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929475 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-lib-modules\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.929568 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.929495 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/891604c5-ed92-44e4-8287-e49e84460dd8-proc\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:39.937489 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:39.937460 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmjj2\" (UniqueName: \"kubernetes.io/projected/891604c5-ed92-44e4-8287-e49e84460dd8-kube-api-access-mmjj2\") pod \"perf-node-gather-daemonset-vxhn4\" (UID: \"891604c5-ed92-44e4-8287-e49e84460dd8\") " pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:40.072119 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:40.072061 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:40.211641 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:40.211604 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4"] Jun 11 13:48:40.214144 ip-10-0-141-252 kubenswrapper[2563]: W0611 13:48:40.214117 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod891604c5_ed92_44e4_8287_e49e84460dd8.slice/crio-8260c14dc93fc436d501c1fe4c20bddac2485b4664818aec88d64eb8d7d2af00 WatchSource:0}: Error finding container 8260c14dc93fc436d501c1fe4c20bddac2485b4664818aec88d64eb8d7d2af00: Status 404 returned error can't find the container with id 8260c14dc93fc436d501c1fe4c20bddac2485b4664818aec88d64eb8d7d2af00 Jun 11 13:48:40.440559 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:40.440534 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6bbwn_2d7fa167-12b1-4e91-93e6-0722528075d2/dns/0.log" Jun 11 13:48:40.464297 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:40.464278 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6bbwn_2d7fa167-12b1-4e91-93e6-0722528075d2/kube-rbac-proxy/0.log" Jun 11 13:48:40.590764 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:40.590744 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5msxz_e504aab2-b809-4e56-8f01-57513a7af2b6/dns-node-resolver/0.log" Jun 11 13:48:41.090919 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:41.090891 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" event={"ID":"891604c5-ed92-44e4-8287-e49e84460dd8","Type":"ContainerStarted","Data":"7fd97c7f6915ec601976edc340befb282a892348aa10849b0d0dff043ab389ea"} Jun 11 13:48:41.091304 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:41.090940 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" event={"ID":"891604c5-ed92-44e4-8287-e49e84460dd8","Type":"ContainerStarted","Data":"8260c14dc93fc436d501c1fe4c20bddac2485b4664818aec88d64eb8d7d2af00"} Jun 11 13:48:41.091304 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:41.090961 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:41.109859 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:41.109813 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" podStartSLOduration=2.109798682 podStartE2EDuration="2.109798682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 13:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 13:48:41.1080735 +0000 UTC m=+2284.351151353" watchObservedRunningTime="2026-06-11 13:48:41.109798682 +0000 UTC m=+2284.352876517" Jun 11 13:48:41.236091 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:41.236065 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-55b4b6869f-frrp6_d3b8ca2e-bc93-4231-8915-80e0457a23c6/registry/0.log" Jun 11 13:48:41.312958 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:41.312936 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qqwrv_90114819-e819-4921-9de2-a9260098a343/node-ca/0.log" Jun 11 13:48:42.352233 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:42.352196 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-z57z9_691275c6-2abb-47a8-ab09-dccc20061027/discovery/0.log" Jun 11 13:48:42.371487 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:42.371463 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f55b6cf49-4btt9_9106ac49-59aa-4d34-8443-10d1d72aa60f/kube-auth-proxy/0.log" Jun 11 13:48:42.492359 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:42.492323 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8swt6_f0147396-b9d6-446f-acc0-6c15687359e8/payload-pre-processing/0.log" Jun 11 13:48:43.210746 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:43.210722 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t294z_811a9938-4e3f-45fc-8c18-a7e91fc04dc8/serve-healthcheck-canary/0.log" Jun 11 13:48:43.861236 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:43.861203 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjjbn_78ee82ec-97dc-4224-b1e8-6dc424124f41/kube-rbac-proxy/0.log" Jun 11 13:48:43.882111 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:43.882082 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjjbn_78ee82ec-97dc-4224-b1e8-6dc424124f41/exporter/0.log" Jun 11 13:48:43.903433 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:43.903406 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjjbn_78ee82ec-97dc-4224-b1e8-6dc424124f41/extractor/0.log" Jun 11 13:48:46.129689 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:46.129657 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-68c659989f-ghq4z_a9326b61-159d-4b6e-9e32-9a015a1e2182/nginx/0.log" Jun 11 13:48:46.177330 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:46.177306 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-tdlg2_3638921d-7768-489d-988f-87e9a06e5b64/manager/0.log" Jun 11 13:48:46.289735 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:46.289712 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-z2hqn_82a9297a-b0c2-43f5-bab1-753963757bd4/manager/0.log" Jun 11 13:48:46.513482 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:46.513458 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cdf5bf854-7nccn_95965310-296e-4f56-b9e0-72f7690f2a3f/manager/0.log" Jun 11 13:48:46.533888 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:46.533865 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cdf5bf854-7nccn_95965310-296e-4f56-b9e0-72f7690f2a3f/copy-manifests/0.log" Jun 11 13:48:47.107396 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:47.107366 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5gk42/perf-node-gather-daemonset-vxhn4" Jun 11 13:48:53.475446 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:53.475411 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-z57z9_691275c6-2abb-47a8-ab09-dccc20061027/discovery/0.log" Jun 11 13:48:53.519943 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:53.519911 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-h8r8n_e72d2172-73fa-4df4-8f86-8bd19ce68941/migrator/0.log" Jun 11 13:48:53.529450 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:53.529431 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f55b6cf49-4btt9_9106ac49-59aa-4d34-8443-10d1d72aa60f/kube-auth-proxy/0.log" Jun 11 13:48:53.561611 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:53.561589 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-h8r8n_e72d2172-73fa-4df4-8f86-8bd19ce68941/graceful-termination/0.log" Jun 11 13:48:53.640781 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:53.640760 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8swt6_f0147396-b9d6-446f-acc0-6c15687359e8/payload-pre-processing/0.log" Jun 11 13:48:54.862367 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:54.862343 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-z57z9_691275c6-2abb-47a8-ab09-dccc20061027/discovery/0.log" Jun 11 13:48:54.872807 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:54.872784 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f55b6cf49-4btt9_9106ac49-59aa-4d34-8443-10d1d72aa60f/kube-auth-proxy/0.log" Jun 11 13:48:54.965304 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:54.965281 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8swt6_f0147396-b9d6-446f-acc0-6c15687359e8/payload-pre-processing/0.log" Jun 11 13:48:55.623723 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.623693 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sgntk_02f763af-9de2-4dbe-91b0-929a96b0ef17/kube-multus-additional-cni-plugins/0.log" Jun 11 13:48:55.649110 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.649080 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sgntk_02f763af-9de2-4dbe-91b0-929a96b0ef17/egress-router-binary-copy/0.log" Jun 11 13:48:55.678954 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.678933 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sgntk_02f763af-9de2-4dbe-91b0-929a96b0ef17/cni-plugins/0.log" Jun 11 13:48:55.701232 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.701212 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sgntk_02f763af-9de2-4dbe-91b0-929a96b0ef17/bond-cni-plugin/0.log" Jun 11 13:48:55.720986 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.720968 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sgntk_02f763af-9de2-4dbe-91b0-929a96b0ef17/routeoverride-cni/0.log" Jun 11 13:48:55.740822 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.740805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sgntk_02f763af-9de2-4dbe-91b0-929a96b0ef17/whereabouts-cni-bincopy/0.log" Jun 11 13:48:55.760584 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.760568 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sgntk_02f763af-9de2-4dbe-91b0-929a96b0ef17/whereabouts-cni/0.log" Jun 11 13:48:55.828077 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.828046 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtgr8_0a32166a-f652-4a18-8c92-5c38f0868a36/kube-multus/0.log" Jun 11 13:48:55.900136 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.900051 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lwnxx_9364f758-a646-4d6c-902a-9e361a572682/network-metrics-daemon/0.log" Jun 11 13:48:55.918912 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:55.918891 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lwnxx_9364f758-a646-4d6c-902a-9e361a572682/kube-rbac-proxy/0.log" Jun 11 13:48:56.689781 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:56.689751 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-z57z9_691275c6-2abb-47a8-ab09-dccc20061027/discovery/0.log" Jun 11 13:48:56.699442 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:56.699420 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f55b6cf49-4btt9_9106ac49-59aa-4d34-8443-10d1d72aa60f/kube-auth-proxy/0.log" Jun 11 13:48:56.787697 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:56.787668 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8swt6_f0147396-b9d6-446f-acc0-6c15687359e8/payload-pre-processing/0.log" Jun 11 13:48:57.142228 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:57.142148 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxgg2_46db2288-b49b-45b5-a190-2f8946b9ae70/ovn-controller/0.log" Jun 11 13:48:57.157299 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:57.157277 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxgg2_46db2288-b49b-45b5-a190-2f8946b9ae70/ovn-acl-logging/0.log" Jun 11 13:48:57.180100 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:57.180075 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxgg2_46db2288-b49b-45b5-a190-2f8946b9ae70/kube-rbac-proxy-node/0.log" Jun 11 13:48:57.205437 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:57.205416 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxgg2_46db2288-b49b-45b5-a190-2f8946b9ae70/kube-rbac-proxy-ovn-metrics/0.log" Jun 11 13:48:57.224484 ip-10-0-141-252 kubenswrapper[2563]: I0611 13:48:57.224465 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxgg2_46db2288-b49b-45b5-a190-2f8946b9ae70/northd/0.log"