Jun 02 01:20:13.921176 ip-10-0-133-218 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 02 01:20:13.921186 ip-10-0-133-218 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 02 01:20:13.921193 ip-10-0-133-218 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 02 01:20:13.921412 ip-10-0-133-218 systemd[1]: Failed to start Kubernetes Kubelet. Jun 02 01:20:24.011151 ip-10-0-133-218 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jun 02 01:20:24.011167 ip-10-0-133-218 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c067a9d37d504381bf2542279ce365d1 -- Jun 02 01:22:51.680931 ip-10-0-133-218 systemd[1]: Starting Kubernetes Kubelet... Jun 02 01:22:52.153377 ip-10-0-133-218 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 01:22:52.153377 ip-10-0-133-218 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 01:22:52.153377 ip-10-0-133-218 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 01:22:52.153377 ip-10-0-133-218 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 01:22:52.153377 ip-10-0-133-218 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 01:22:52.155085 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.154998 2564 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 01:22:52.160701 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160677 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 01:22:52.160701 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160697 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 01:22:52.160701 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160702 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 01:22:52.160701 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160706 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160710 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160714 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160718 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160724 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160728 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160732 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160736 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160740 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160744 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160748 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160752 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160756 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160762 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160769 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160773 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160778 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160783 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160786 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160791 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 01:22:52.160872 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160795 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160798 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160803 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160807 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160811 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160816 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160819 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160823 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160827 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160832 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160835 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160837 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160840 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160844 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160848 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160851 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160854 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160856 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160859 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 01:22:52.161356 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160861 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160864 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160866 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160869 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160872 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160875 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160877 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160880 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160882 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160885 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160887 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160890 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160893 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160895 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160898 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160900 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160903 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160906 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160908 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160911 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 01:22:52.161841 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160914 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160916 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160919 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160922 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160924 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160927 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160931 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160934 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160936 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160948 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160952 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160955 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160957 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160960 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160963 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160965 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160978 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160981 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160984 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 01:22:52.162341 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160986 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160989 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160991 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160994 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.160996 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161372 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161377 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161379 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161383 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161385 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161388 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161390 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161393 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161396 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161399 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161401 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161404 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161407 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161409 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 01:22:52.162829 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161413 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161416 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161420 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161423 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161425 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161428 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161431 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161433 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161436 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161438 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161446 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161449 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161452 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161454 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161456 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161459 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161461 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161464 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161466 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161469 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 01:22:52.163294 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161471 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161474 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161476 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161478 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161481 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161485 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161488 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161490 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161493 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161496 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161498 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161501 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161503 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161506 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161509 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161511 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161514 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161516 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161518 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161521 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 01:22:52.163807 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161523 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161526 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161528 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161538 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161542 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161545 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161547 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161550 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161553 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161556 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161558 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161560 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161563 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161565 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161568 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161570 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161573 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161576 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161578 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 01:22:52.164298 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161580 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161583 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161585 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161588 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161591 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161594 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161597 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161600 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161602 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161605 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161607 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161610 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.161612 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162813 2564 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162823 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162833 2564 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162838 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162848 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162852 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162856 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162860 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 01:22:52.164778 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162863 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162866 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162870 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162873 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162876 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162879 2564 flags.go:64] FLAG: --cgroup-root="" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162882 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162885 2564 flags.go:64] FLAG: --client-ca-file="" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162888 2564 flags.go:64] FLAG: --cloud-config="" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162891 2564 flags.go:64] FLAG: --cloud-provider="external" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162893 2564 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162900 2564 flags.go:64] FLAG: --cluster-domain="" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162902 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162905 2564 flags.go:64] FLAG: --config-dir="" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162908 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162911 2564 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162915 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162918 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162921 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162924 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162927 2564 flags.go:64] FLAG: --contention-profiling="false" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162930 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162933 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162936 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162939 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 01:22:52.165303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162943 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162946 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162949 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162952 2564 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162962 2564 flags.go:64] FLAG: --enable-server="true" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162965 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162972 2564 flags.go:64] FLAG: --event-burst="100" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162975 2564 flags.go:64] FLAG: --event-qps="50" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162978 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162981 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162984 2564 flags.go:64] FLAG: --eviction-hard="" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162988 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162992 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162994 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.162998 2564 flags.go:64] FLAG: --eviction-soft="" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163001 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163004 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163007 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163010 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163013 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163016 2564 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163018 2564 flags.go:64] FLAG: --feature-gates="" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163022 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163025 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163028 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 01:22:52.165950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163031 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163034 2564 flags.go:64] FLAG: --healthz-port="10248" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163037 2564 flags.go:64] FLAG: --help="false" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163040 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163043 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163046 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163049 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163052 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163056 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163058 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163061 2564 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163064 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163072 2564 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163075 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163078 2564 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163081 2564 flags.go:64] FLAG: --kube-reserved="" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163084 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163087 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163090 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163092 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163095 2564 flags.go:64] FLAG: --lock-file="" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163098 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163102 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163105 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 01:22:52.166579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163110 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163113 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163116 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163119 2564 flags.go:64] FLAG: --logging-format="text" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163121 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163125 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163127 2564 flags.go:64] FLAG: --manifest-url="" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163130 2564 flags.go:64] FLAG: --manifest-url-header="" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163134 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163137 2564 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163141 2564 flags.go:64] FLAG: --max-pods="110" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163144 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163147 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163150 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163153 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163156 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163159 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163161 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163169 2564 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163172 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163175 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163184 2564 flags.go:64] FLAG: --pod-cidr="" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163187 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 01:22:52.167188 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163195 2564 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163197 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163200 2564 flags.go:64] FLAG: --pods-per-core="0" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163203 2564 flags.go:64] FLAG: --port="10250" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163206 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163209 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0def7e88f7b28d2e5" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163212 2564 flags.go:64] FLAG: --qos-reserved="" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163215 2564 flags.go:64] FLAG: --read-only-port="10255" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163218 2564 flags.go:64] FLAG: --register-node="true" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163221 2564 flags.go:64] FLAG: --register-schedulable="true" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163224 2564 flags.go:64] FLAG: --register-with-taints="" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163228 2564 flags.go:64] FLAG: --registry-burst="10" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163230 2564 flags.go:64] FLAG: --registry-qps="5" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163233 2564 flags.go:64] FLAG: --reserved-cpus="" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163236 2564 flags.go:64] FLAG: --reserved-memory="" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163239 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163243 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163245 2564 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163248 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163251 2564 flags.go:64] FLAG: --runonce="false" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163253 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163256 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163259 2564 flags.go:64] FLAG: --seccomp-default="false" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163262 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163265 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163267 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 01:22:52.167806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163271 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163273 2564 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163276 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163279 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163281 2564 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163290 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163298 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163301 2564 flags.go:64] FLAG: --system-cgroups="" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163303 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163309 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163311 2564 flags.go:64] FLAG: --tls-cert-file="" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163314 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163320 2564 flags.go:64] FLAG: --tls-min-version="" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163323 2564 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163326 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163329 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163333 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163336 2564 flags.go:64] FLAG: --v="2" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163340 2564 flags.go:64] FLAG: --version="false" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163343 2564 flags.go:64] FLAG: --vmodule="" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163347 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163351 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163458 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163461 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 01:22:52.168436 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163464 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163467 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163470 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163473 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163476 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163479 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163482 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163484 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163487 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163490 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163494 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163497 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163500 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163503 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163513 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163516 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163518 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163521 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163523 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 01:22:52.169073 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163526 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163528 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163531 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163534 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163537 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163539 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163542 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163544 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163547 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163549 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163552 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163554 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163557 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163559 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163562 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163564 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163567 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163569 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163572 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163574 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 01:22:52.169554 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163576 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163580 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163583 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163586 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163589 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163592 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163594 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163598 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163608 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163610 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163613 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163615 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163618 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163620 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163623 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163626 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163628 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163631 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163633 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 01:22:52.170065 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163636 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163638 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163640 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163643 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163645 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163648 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163664 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163667 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163670 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163672 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163674 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163677 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163680 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163683 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163685 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163688 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163690 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163693 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163695 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163698 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 01:22:52.170580 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163702 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163704 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163713 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163716 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163719 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.163721 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.163726 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.170687 2564 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.170705 2564 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170750 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170755 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170758 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170761 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170764 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170766 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170769 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 01:22:52.171196 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170772 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170775 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170777 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170780 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170782 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170786 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170791 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170796 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170799 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170802 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170805 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170808 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170811 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170813 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170816 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170818 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170821 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170824 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170826 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 01:22:52.171603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170829 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170831 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170834 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170836 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170839 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170843 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170847 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170849 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170852 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170855 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170857 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170860 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170862 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170865 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170867 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170870 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170872 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170875 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170877 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170880 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 01:22:52.172086 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170882 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170885 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170887 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170890 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170893 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170895 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170898 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170901 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170903 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170906 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170908 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170911 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170914 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170916 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170919 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170921 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170924 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170927 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170931 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170934 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 01:22:52.172599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170937 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170939 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170942 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170944 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170947 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170949 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170952 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170955 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170957 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170960 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170962 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170965 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170967 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170969 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170972 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170975 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170977 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170980 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170983 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 01:22:52.173133 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.170985 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.170990 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171111 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171115 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171119 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171122 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171126 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171129 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171131 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171134 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171138 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171141 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171144 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171147 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171150 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171153 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 01:22:52.173603 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171155 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171158 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171161 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171163 2564 feature_gate.go:328] unrecognized feature gate: Example Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171166 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171168 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171171 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171175 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171178 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171181 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171184 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171186 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171189 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171192 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171195 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171198 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171200 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171203 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171206 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 01:22:52.174026 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171209 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171211 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171214 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171216 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171219 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171222 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171224 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171227 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171230 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171233 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171235 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171238 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171240 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171243 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171247 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171250 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171252 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171255 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171257 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171259 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 01:22:52.174478 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171262 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171264 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171267 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171269 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171272 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171274 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171277 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171279 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171282 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171284 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171287 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171289 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171292 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171294 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171296 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171299 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171301 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171304 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171306 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171309 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 01:22:52.174977 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171311 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171314 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171316 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171319 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171322 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171324 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171327 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171330 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171332 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171335 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171338 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171340 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:52.171343 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.171347 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 01:22:52.175460 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.172141 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 01:22:52.176465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.176451 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 01:22:52.177392 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.177381 2564 server.go:1019] "Starting client certificate rotation" Jun 02 01:22:52.177498 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.177480 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 01:22:52.177529 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.177518 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 01:22:52.201077 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.201058 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 01:22:52.202920 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.202903 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 01:22:52.213739 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.213722 2564 log.go:25] "Validated CRI v1 runtime API" Jun 02 01:22:52.222686 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.222670 2564 log.go:25] "Validated CRI v1 image API" Jun 02 01:22:52.225081 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.225068 2564 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 01:22:52.228989 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.228965 2564 fs.go:135] Filesystem UUIDs: map[4457f2eb-486f-46e4-8056-ae3ce8ec8e7e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e8fdc6d1-ff5b-4890-870d-66bce5d9dbb2:/dev/nvme0n1p3] Jun 02 01:22:52.228989 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.228983 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 01:22:52.233782 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.233767 2564 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 01:22:52.234023 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.233925 2564 manager.go:217] Machine: {Timestamp:2026-06-02 01:22:52.23338091 +0000 UTC m=+0.437841182 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102558 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e123b5b2241cdccb28aeefeb3d754 SystemUUID:ec2e123b-5b22-41cd-ccb2-8aeefeb3d754 BootID:c067a9d3-7d50-4381-bf25-42279ce365d1 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bd:b8:91:8f:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bd:b8:91:8f:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:eb:87:98:e2:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 01:22:52.234090 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.234045 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 01:22:52.234152 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.234141 2564 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 01:22:52.235461 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.235441 2564 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 01:22:52.235601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.235464 2564 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-218.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 01:22:52.235663 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.235610 2564 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 01:22:52.235663 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.235619 2564 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 01:22:52.235663 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.235627 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 01:22:52.237038 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.237025 2564 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 01:22:52.238268 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.238259 2564 state_mem.go:36] "Initialized new in-memory state store" Jun 02 01:22:52.238383 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.238374 2564 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 01:22:52.241194 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.241185 2564 kubelet.go:491] "Attempting to sync node with API server" Jun 02 01:22:52.241234 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.241203 2564 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 01:22:52.241234 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.241214 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 01:22:52.241234 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.241223 2564 kubelet.go:397] "Adding apiserver pod source" Jun 02 01:22:52.241329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.241237 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 01:22:52.242466 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.242455 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 01:22:52.242504 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.242473 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 01:22:52.245856 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.245837 2564 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 01:22:52.248326 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.248310 2564 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 01:22:52.251215 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251200 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251219 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251229 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251236 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251249 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251257 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251266 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251274 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251284 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251293 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251305 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 01:22:52.251301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.251313 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 01:22:52.252909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.252889 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 01:22:52.252909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.252913 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 01:22:52.256214 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.255925 2564 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-218.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 01:22:52.256324 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.256275 2564 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 01:22:52.256432 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.256280 2564 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 01:22:52.258187 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.258155 2564 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 01:22:52.258479 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.258465 2564 server.go:1295] "Started kubelet" Jun 02 01:22:52.258568 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.258526 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 01:22:52.258716 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.258682 2564 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 01:22:52.258755 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.258735 2564 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 01:22:52.259299 ip-10-0-133-218 systemd[1]: Started Kubernetes Kubelet. Jun 02 01:22:52.260191 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.260140 2564 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 01:22:52.261323 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.261309 2564 server.go:317] "Adding debug handlers to kubelet server" Jun 02 01:22:52.261730 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.261709 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fszbq" Jun 02 01:22:52.267195 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267181 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 01:22:52.267276 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267196 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 01:22:52.267276 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267265 2564 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 01:22:52.267276 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267273 2564 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 01:22:52.267404 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267276 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fszbq" Jun 02 01:22:52.267404 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267323 2564 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 01:22:52.267497 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267412 2564 reconstruct.go:97] "Volume reconstruction finished" Jun 02 01:22:52.267497 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267420 2564 reconciler.go:26] "Reconciler: start to sync state" Jun 02 01:22:52.267915 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267897 2564 factory.go:55] Registering systemd factory Jun 02 01:22:52.267915 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.267916 2564 factory.go:223] Registration of the systemd container factory successfully Jun 02 01:22:52.268066 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.268045 2564 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 01:22:52.268119 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.268107 2564 factory.go:153] Registering CRI-O factory Jun 02 01:22:52.268162 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.268121 2564 factory.go:223] Registration of the crio container factory successfully Jun 02 01:22:52.268216 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.268167 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 01:22:52.268216 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.268185 2564 factory.go:103] Registering Raw factory Jun 02 01:22:52.268216 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.268200 2564 manager.go:1196] Started watching for new ooms in manager Jun 02 01:22:52.268680 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.268668 2564 manager.go:319] Starting recovery of all containers Jun 02 01:22:52.268812 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.268792 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:52.274235 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.274076 2564 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 01:22:52.277889 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.277860 2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-218.ec2.internal\" not found" node="ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.281061 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.281043 2564 manager.go:324] Recovery completed Jun 02 01:22:52.282795 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.282762 2564 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 02 01:22:52.286319 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.286308 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 01:22:52.289241 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.289222 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientMemory" Jun 02 01:22:52.289329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.289253 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 01:22:52.289329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.289263 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientPID" Jun 02 01:22:52.289807 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.289789 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 01:22:52.289807 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.289803 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 01:22:52.289960 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.289825 2564 state_mem.go:36] "Initialized new in-memory state store" Jun 02 01:22:52.293370 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.293356 2564 policy_none.go:49] "None policy: Start" Jun 02 01:22:52.293370 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.293372 2564 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 01:22:52.293465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.293383 2564 state_mem.go:35] "Initializing new in-memory state store" Jun 02 01:22:52.324223 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.324211 2564 manager.go:341] "Starting Device Plugin manager" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.324237 2564 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.324247 2564 server.go:85] "Starting device plugin registration server" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.324474 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.324485 2564 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.324613 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.324692 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.324700 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.325794 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 01:22:52.338876 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.325830 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:52.371368 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.371339 2564 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 01:22:52.372492 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.372476 2564 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 01:22:52.372556 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.372507 2564 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 01:22:52.372556 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.372525 2564 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 01:22:52.372556 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.372531 2564 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 01:22:52.372720 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.372570 2564 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 01:22:52.376864 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.376845 2564 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 01:22:52.425579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.425533 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 01:22:52.426504 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.426491 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientMemory" Jun 02 01:22:52.426569 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.426519 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 01:22:52.426569 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.426529 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientPID" Jun 02 01:22:52.426569 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.426552 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.433517 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.433503 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.433561 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.433524 2564 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-218.ec2.internal\": node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:52.455706 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.455689 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:52.473212 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.473192 2564 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal"] Jun 02 01:22:52.473278 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.473248 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 01:22:52.474727 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.474713 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientMemory" Jun 02 01:22:52.474802 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.474738 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 01:22:52.474802 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.474751 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientPID" Jun 02 01:22:52.476078 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476067 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 01:22:52.476220 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476207 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.476269 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476234 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 01:22:52.476705 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476692 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientMemory" Jun 02 01:22:52.476792 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476705 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientMemory" Jun 02 01:22:52.476792 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476720 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 01:22:52.476792 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476725 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 01:22:52.476792 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476735 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientPID" Jun 02 01:22:52.476792 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.476740 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientPID" Jun 02 01:22:52.477845 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.477828 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.477934 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.477858 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 01:22:52.478470 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.478456 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientMemory" Jun 02 01:22:52.478543 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.478479 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 01:22:52.478543 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.478489 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeHasSufficientPID" Jun 02 01:22:52.495164 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.495143 2564 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-218.ec2.internal\" not found" node="ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.498439 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.498424 2564 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-218.ec2.internal\" not found" node="ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.555900 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.555881 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:52.655989 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.655967 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:52.669264 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.669235 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f81178f6a81c011305b0e997a8ecfc74-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal\" (UID: \"f81178f6a81c011305b0e997a8ecfc74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.669347 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.669273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f81178f6a81c011305b0e997a8ecfc74-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal\" (UID: \"f81178f6a81c011305b0e997a8ecfc74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.669347 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.669312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/645034bd148c3fd0460f6af9403bc9a0-config\") pod \"kube-apiserver-proxy-ip-10-0-133-218.ec2.internal\" (UID: \"645034bd148c3fd0460f6af9403bc9a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.756682 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.756623 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:52.769943 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.769928 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f81178f6a81c011305b0e997a8ecfc74-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal\" (UID: \"f81178f6a81c011305b0e997a8ecfc74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.769987 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.769950 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f81178f6a81c011305b0e997a8ecfc74-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal\" (UID: \"f81178f6a81c011305b0e997a8ecfc74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.769987 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.769976 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/645034bd148c3fd0460f6af9403bc9a0-config\") pod \"kube-apiserver-proxy-ip-10-0-133-218.ec2.internal\" (UID: \"645034bd148c3fd0460f6af9403bc9a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.770057 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.770001 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/645034bd148c3fd0460f6af9403bc9a0-config\") pod \"kube-apiserver-proxy-ip-10-0-133-218.ec2.internal\" (UID: \"645034bd148c3fd0460f6af9403bc9a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.770057 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.770021 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f81178f6a81c011305b0e997a8ecfc74-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal\" (UID: \"f81178f6a81c011305b0e997a8ecfc74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.770057 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.770026 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f81178f6a81c011305b0e997a8ecfc74-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal\" (UID: \"f81178f6a81c011305b0e997a8ecfc74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.797051 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.797028 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.801483 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:52.801466 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" Jun 02 01:22:52.856965 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.856935 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:52.957420 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:52.957392 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:53.057992 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:53.057927 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:53.093127 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.093107 2564 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 01:22:53.158621 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:53.158598 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:53.177026 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.177010 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 01:22:53.177117 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.177102 2564 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 01:22:53.177168 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.177110 2564 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 01:22:53.177168 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.177134 2564 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 01:22:53.259256 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:53.259228 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-218.ec2.internal\" not found" Jun 02 01:22:53.267863 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.267842 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 01:22:53.269269 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.269236 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 01:17:52 +0000 UTC" deadline="2027-11-21 20:15:57.381276931 +0000 UTC" Jun 02 01:22:53.269351 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.269269 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12906h53m4.112011548s" Jun 02 01:22:53.276614 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.276595 2564 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 01:22:53.299831 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.299813 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4z87" Jun 02 01:22:53.306467 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.306446 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4z87" Jun 02 01:22:53.311589 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.311544 2564 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 01:22:53.368519 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.368495 2564 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" Jun 02 01:22:53.369411 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:53.369384 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645034bd148c3fd0460f6af9403bc9a0.slice/crio-315be65514b3fe692633f0589b596df16f52d1a4a8f64dc9f7e51611faee5126 WatchSource:0}: Error finding container 315be65514b3fe692633f0589b596df16f52d1a4a8f64dc9f7e51611faee5126: Status 404 returned error can't find the container with id 315be65514b3fe692633f0589b596df16f52d1a4a8f64dc9f7e51611faee5126 Jun 02 01:22:53.369599 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:53.369572 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf81178f6a81c011305b0e997a8ecfc74.slice/crio-cc70f90d55dfb2fd91cecab18391d4c17053cd4b5cfd18797929a2a8ccb37b9a WatchSource:0}: Error finding container cc70f90d55dfb2fd91cecab18391d4c17053cd4b5cfd18797929a2a8ccb37b9a: Status 404 returned error can't find the container with id cc70f90d55dfb2fd91cecab18391d4c17053cd4b5cfd18797929a2a8ccb37b9a Jun 02 01:22:53.372786 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.372764 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 01:22:53.376558 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.376521 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" event={"ID":"645034bd148c3fd0460f6af9403bc9a0","Type":"ContainerStarted","Data":"315be65514b3fe692633f0589b596df16f52d1a4a8f64dc9f7e51611faee5126"} Jun 02 01:22:53.377809 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.377789 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" event={"ID":"f81178f6a81c011305b0e997a8ecfc74","Type":"ContainerStarted","Data":"cc70f90d55dfb2fd91cecab18391d4c17053cd4b5cfd18797929a2a8ccb37b9a"} Jun 02 01:22:53.380304 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.380286 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 01:22:53.381444 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.381430 2564 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" Jun 02 01:22:53.387738 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:53.387723 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 01:22:54.088852 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.088821 2564 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 01:22:54.242333 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.242303 2564 apiserver.go:52] "Watching apiserver" Jun 02 01:22:54.249563 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.249540 2564 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 01:22:54.252268 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.252242 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-7tmcl","openshift-network-operator/iptables-alerter-shd9s","openshift-ovn-kubernetes/ovnkube-node-qsmk7","kube-system/global-pull-secret-syncer-4vq4w","kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hbngb","openshift-dns/node-resolver-wn6w8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal","openshift-multus/multus-672sz","openshift-multus/network-metrics-daemon-zvkvb","openshift-network-diagnostics/network-check-target-f7cf5","kube-system/konnectivity-agent-xkf6n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z","openshift-image-registry/node-ca-8lt9m"] Jun 02 01:22:54.254978 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.254833 2564 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 01:22:54.255079 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.254984 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:54.255079 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.255067 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:22:54.256961 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.256906 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:54.256961 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.256974 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:22:54.259005 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.258982 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.260918 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.260893 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.261168 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.261146 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 01:22:54.261440 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.261421 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 01:22:54.261440 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.261432 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 01:22:54.261593 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.261521 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mncp4\"" Jun 02 01:22:54.261794 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.261775 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 01:22:54.261903 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.261803 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 01:22:54.261903 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.261810 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 01:22:54.262955 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.262934 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 01:22:54.263144 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.263124 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:22:54.263321 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.263297 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 01:22:54.263398 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.263268 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 01:22:54.263541 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.263522 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ttvn6\"" Jun 02 01:22:54.265565 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.265204 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 01:22:54.265565 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.265361 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-l2s2w\"" Jun 02 01:22:54.265565 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.265432 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 01:22:54.268000 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.267739 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.270791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.269819 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 01:22:54.270791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.270754 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 01:22:54.270791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.270781 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kknss\"" Jun 02 01:22:54.272510 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.272308 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-672sz" Jun 02 01:22:54.272510 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.272493 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.274508 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.274467 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 01:22:54.274603 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.274467 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 01:22:54.275468 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.274858 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 01:22:54.275468 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.274872 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 01:22:54.275468 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.275008 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mvfc4\"" Jun 02 01:22:54.275468 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.275085 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b4hl4\"" Jun 02 01:22:54.275468 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.275102 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 01:22:54.275468 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.275175 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 01:22:54.275468 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.275223 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 01:22:54.276329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.276141 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-multus-certs\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.276329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.276171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-cni-bin\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.276329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.276195 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-env-overrides\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.276329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.276222 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-ovn\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.276329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.276250 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvnzz\" (UniqueName: \"kubernetes.io/projected/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-kube-api-access-gvnzz\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:54.276329 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.276302 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-conf-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.276624 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.276377 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chs2v\" (UniqueName: \"kubernetes.io/projected/a415cede-b0c5-4300-8f99-22397439fe3f-kube-api-access-chs2v\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.277344 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.276794 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.277344 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277002 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.277490 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-cni-netd\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.277490 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277457 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-ovnkube-script-lib\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.277490 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277487 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-socket-dir-parent\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.277631 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277511 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-systemd-units\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.277631 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277562 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-systemd\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.277631 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-tuned\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.277786 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52b5b39e-4c8a-48ed-beae-191f194d012b-konnectivity-ca\") pod \"konnectivity-agent-xkf6n\" (UID: \"52b5b39e-4c8a-48ed-beae-191f194d012b\") " pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:22:54.277786 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-system-cni-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.277786 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277716 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-cni-bin\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.277924 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-kubelet\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.277924 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277855 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.277924 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277903 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-ovnkube-config\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.278068 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.277927 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysctl-conf\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.278068 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-cni-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.278068 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278056 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-modprobe-d\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.278199 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278081 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-var-lib-kubelet\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.278199 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278101 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a415cede-b0c5-4300-8f99-22397439fe3f-host\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.278199 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278131 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52b5b39e-4c8a-48ed-beae-191f194d012b-agent-certs\") pod \"konnectivity-agent-xkf6n\" (UID: \"52b5b39e-4c8a-48ed-beae-191f194d012b\") " pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:22:54.278199 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278163 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-netns\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.278381 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278197 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-etc-kubernetes\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.278381 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278234 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-log-socket\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.278381 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278281 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-host\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.278381 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278304 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-k8s-cni-cncf-io\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.278381 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278339 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-lib-modules\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278380 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77bjb\" (UniqueName: \"kubernetes.io/projected/aa180230-fe8b-48c0-be17-f47f8d9be3bc-kube-api-access-77bjb\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-cni-multus\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278438 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-cnibin\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278472 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/401e1384-3579-4929-86af-eb642d871287-multus-daemon-config\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-kubelet\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278540 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-slash\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278565 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-run-netns\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278591 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-node-log\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.278626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278616 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/962fbf55-761d-4399-9cd3-6d3cfb488044-ovn-node-metrics-cert\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-kubernetes\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278681 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a415cede-b0c5-4300-8f99-22397439fe3f-serviceca\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-os-release\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278725 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/401e1384-3579-4929-86af-eb642d871287-cni-binary-copy\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278769 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysconfig\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278800 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-run\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278828 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjw67\" (UniqueName: \"kubernetes.io/projected/401e1384-3579-4929-86af-eb642d871287-kube-api-access-bjw67\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278852 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-systemd\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-var-lib-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278914 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysctl-d\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278984 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-hostroot\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.278994 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279001 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-etc-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.279047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279065 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279089 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-run-ovn-kubernetes\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279133 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfcpm\" (UniqueName: \"kubernetes.io/projected/962fbf55-761d-4399-9cd3-6d3cfb488044-kube-api-access-sfcpm\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279163 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-sys\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279195 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa180230-fe8b-48c0-be17-f47f8d9be3bc-tmp\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279227 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279257 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-72fck\"" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279281 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279291 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j86jz\"" Jun 02 01:22:54.279751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.279227 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 01:22:54.281200 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.281183 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.281274 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.281212 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.281330 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.281269 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:22:54.283522 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.283405 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 01:22:54.283522 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.283436 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ncvtd\"" Jun 02 01:22:54.284057 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.284038 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 01:22:54.307186 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.307113 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 01:17:53 +0000 UTC" deadline="2027-11-10 10:44:20.670688732 +0000 UTC" Jun 02 01:22:54.307186 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.307141 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12633h21m26.363550978s" Jun 02 01:22:54.368489 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.368460 2564 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 01:22:54.379968 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.379940 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.380122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.379983 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysctl-d\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.380122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-sys-fs\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.380122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380045 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-hostroot\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.380122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380068 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-etc-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.380122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380085 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ctxc\" (UniqueName: \"kubernetes.io/projected/104ed972-6629-4e74-82b9-81e9dccec6bc-kube-api-access-9ctxc\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.380122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380101 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-system-cni-dir\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.380122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380121 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-cnibin\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380162 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380197 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380205 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-etc-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380225 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-hostroot\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-kubelet-config\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380306 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-dbus\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380335 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380342 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysctl-d\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380363 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:54.380446 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.380446 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380487 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-run-ovn-kubernetes\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380517 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfcpm\" (UniqueName: \"kubernetes.io/projected/962fbf55-761d-4399-9cd3-6d3cfb488044-kube-api-access-sfcpm\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380530 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-run-ovn-kubernetes\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.380607 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs podName:cf148b2d-3a22-4a94-98f3-5dcc28eebd87 nodeName:}" failed. No retries permitted until 2026-06-02 01:22:54.880583143 +0000 UTC m=+3.085043410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs") pod "network-metrics-daemon-zvkvb" (UID: "cf148b2d-3a22-4a94-98f3-5dcc28eebd87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380610 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-sys\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380791 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa180230-fe8b-48c0-be17-f47f8d9be3bc-tmp\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380858 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/104ed972-6629-4e74-82b9-81e9dccec6bc-host-slash\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380797 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-sys\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.380909 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380889 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-multus-certs\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380929 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-multus-certs\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380927 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-cni-bin\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.380973 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-env-overrides\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381029 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-ovn\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/234e4e76-252c-491b-b14f-6b9a7e2c36b9-tmp-dir\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381062 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-cni-bin\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-os-release\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381112 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381133 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gvnzz\" (UniqueName: \"kubernetes.io/projected/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-kube-api-access-gvnzz\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381159 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-conf-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381206 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-chs2v\" (UniqueName: \"kubernetes.io/projected/a415cede-b0c5-4300-8f99-22397439fe3f-kube-api-access-chs2v\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381230 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-cni-netd\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381282 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-ovnkube-script-lib\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.381327 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381317 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-socket-dir-parent\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381289 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-ovn\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381342 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-systemd-units\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381380 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-systemd-units\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381397 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-systemd\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381427 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-tuned\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381445 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-conf-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381319 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-cni-netd\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381469 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/104ed972-6629-4e74-82b9-81e9dccec6bc-iptables-alerter-script\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381437 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-socket-dir-parent\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-registration-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-device-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381539 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-systemd\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381557 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52b5b39e-4c8a-48ed-beae-191f194d012b-konnectivity-ca\") pod \"konnectivity-agent-xkf6n\" (UID: \"52b5b39e-4c8a-48ed-beae-191f194d012b\") " pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381584 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-system-cni-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381608 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-cni-bin\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381632 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-kubelet\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381677 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-env-overrides\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381709 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-system-cni-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381721 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381679 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381807 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-cni-bin\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381809 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-ovnkube-config\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381843 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysctl-conf\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/234e4e76-252c-491b-b14f-6b9a7e2c36b9-hosts-file\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381897 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-cni-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381921 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-modprobe-d\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381943 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-var-lib-kubelet\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-socket-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-etc-selinux\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382041 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc9bq\" (UniqueName: \"kubernetes.io/projected/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-kube-api-access-nc9bq\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382067 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a415cede-b0c5-4300-8f99-22397439fe3f-host\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382090 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52b5b39e-4c8a-48ed-beae-191f194d012b-agent-certs\") pod \"konnectivity-agent-xkf6n\" (UID: \"52b5b39e-4c8a-48ed-beae-191f194d012b\") " pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382114 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-netns\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.382979 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382137 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-etc-kubernetes\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382163 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52b5b39e-4c8a-48ed-beae-191f194d012b-konnectivity-ca\") pod \"konnectivity-agent-xkf6n\" (UID: \"52b5b39e-4c8a-48ed-beae-191f194d012b\") " pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382175 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-log-socket\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.381777 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-kubelet\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382207 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-host\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382218 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-log-socket\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382230 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-k8s-cni-cncf-io\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382245 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-ovnkube-config\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382275 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-k8s-cni-cncf-io\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382281 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-host\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382341 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-run-netns\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382376 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-etc-kubernetes\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382419 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-lib-modules\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382452 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-77bjb\" (UniqueName: \"kubernetes.io/projected/aa180230-fe8b-48c0-be17-f47f8d9be3bc-kube-api-access-77bjb\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382456 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-modprobe-d\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382483 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382491 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-var-lib-kubelet\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382501 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysctl-conf\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.383781 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382517 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k464m\" (UniqueName: \"kubernetes.io/projected/e85ff609-4afc-446a-83cd-77f70781dd24-kube-api-access-k464m\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382567 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-cni-multus\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382590 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-lib-modules\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382600 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-cnibin\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382622 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/401e1384-3579-4929-86af-eb642d871287-multus-daemon-config\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382629 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a415cede-b0c5-4300-8f99-22397439fe3f-host\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382644 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-kubelet\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382644 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-host-var-lib-cni-multus\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382713 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-slash\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382739 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-run-netns\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382756 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-kubelet\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382774 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-cnibin\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382806 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382823 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-run-netns\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382840 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-node-log\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382867 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-host-slash\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382867 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/962fbf55-761d-4399-9cd3-6d3cfb488044-ovn-node-metrics-cert\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382910 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-node-log\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.384601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382920 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-kubernetes\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382950 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.382977 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a415cede-b0c5-4300-8f99-22397439fe3f-serviceca\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383011 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-os-release\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383040 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/401e1384-3579-4929-86af-eb642d871287-cni-binary-copy\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383064 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysconfig\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383187 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-sysconfig\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383227 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-run\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383278 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr8jh\" (UniqueName: \"kubernetes.io/projected/234e4e76-252c-491b-b14f-6b9a7e2c36b9-kube-api-access-nr8jh\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383299 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/401e1384-3579-4929-86af-eb642d871287-multus-daemon-config\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383318 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjw67\" (UniqueName: \"kubernetes.io/projected/401e1384-3579-4929-86af-eb642d871287-kube-api-access-bjw67\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383345 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-systemd\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383356 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-kubernetes\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383373 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-var-lib-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383385 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a415cede-b0c5-4300-8f99-22397439fe3f-serviceca\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383428 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-os-release\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383475 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa180230-fe8b-48c0-be17-f47f8d9be3bc-run\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383512 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-var-lib-openvswitch\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.385514 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383517 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/401e1384-3579-4929-86af-eb642d871287-multus-cni-dir\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.386372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383516 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/401e1384-3579-4929-86af-eb642d871287-cni-binary-copy\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.386372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383555 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/962fbf55-761d-4399-9cd3-6d3cfb488044-run-systemd\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.386372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.383617 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/962fbf55-761d-4399-9cd3-6d3cfb488044-ovnkube-script-lib\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.386372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.385830 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa180230-fe8b-48c0-be17-f47f8d9be3bc-etc-tuned\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.386372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.386053 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa180230-fe8b-48c0-be17-f47f8d9be3bc-tmp\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.386372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.386353 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52b5b39e-4c8a-48ed-beae-191f194d012b-agent-certs\") pod \"konnectivity-agent-xkf6n\" (UID: \"52b5b39e-4c8a-48ed-beae-191f194d012b\") " pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:22:54.388138 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.387395 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 01:22:54.388138 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.387417 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 01:22:54.388138 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.387428 2564 projected.go:194] Error preparing data for projected volume kube-api-access-pxnvl for pod openshift-network-diagnostics/network-check-target-f7cf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:22:54.388138 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.387552 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl podName:16ba9005-eca6-42a7-96ab-800c12788d7b nodeName:}" failed. No retries permitted until 2026-06-02 01:22:54.88753543 +0000 UTC m=+3.091995706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pxnvl" (UniqueName: "kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl") pod "network-check-target-f7cf5" (UID: "16ba9005-eca6-42a7-96ab-800c12788d7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:22:54.388138 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.387601 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/962fbf55-761d-4399-9cd3-6d3cfb488044-ovn-node-metrics-cert\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.390553 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.390517 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfcpm\" (UniqueName: \"kubernetes.io/projected/962fbf55-761d-4399-9cd3-6d3cfb488044-kube-api-access-sfcpm\") pod \"ovnkube-node-qsmk7\" (UID: \"962fbf55-761d-4399-9cd3-6d3cfb488044\") " pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.391531 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.390800 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvnzz\" (UniqueName: \"kubernetes.io/projected/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-kube-api-access-gvnzz\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:54.391531 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.390913 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chs2v\" (UniqueName: \"kubernetes.io/projected/a415cede-b0c5-4300-8f99-22397439fe3f-kube-api-access-chs2v\") pod \"node-ca-8lt9m\" (UID: \"a415cede-b0c5-4300-8f99-22397439fe3f\") " pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.392331 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.392282 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjw67\" (UniqueName: \"kubernetes.io/projected/401e1384-3579-4929-86af-eb642d871287-kube-api-access-bjw67\") pod \"multus-672sz\" (UID: \"401e1384-3579-4929-86af-eb642d871287\") " pod="openshift-multus/multus-672sz" Jun 02 01:22:54.392550 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.392485 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77bjb\" (UniqueName: \"kubernetes.io/projected/aa180230-fe8b-48c0-be17-f47f8d9be3bc-kube-api-access-77bjb\") pod \"tuned-hbngb\" (UID: \"aa180230-fe8b-48c0-be17-f47f8d9be3bc\") " pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.483867 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.483834 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484014 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.483879 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484014 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.483913 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nr8jh\" (UniqueName: \"kubernetes.io/projected/234e4e76-252c-491b-b14f-6b9a7e2c36b9-kube-api-access-nr8jh\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.484014 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.483945 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-sys-fs\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.484014 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.483973 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9ctxc\" (UniqueName: \"kubernetes.io/projected/104ed972-6629-4e74-82b9-81e9dccec6bc-kube-api-access-9ctxc\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.484228 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484166 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-sys-fs\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.484228 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484203 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-system-cni-dir\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484323 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484232 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-cnibin\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484323 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484257 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484323 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484283 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-kubelet-config\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.484475 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484352 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-kubelet-config\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.484475 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484396 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-system-cni-dir\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484475 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484398 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-cnibin\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484475 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484422 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-dbus\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.484475 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484436 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484492 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/104ed972-6629-4e74-82b9-81e9dccec6bc-host-slash\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484528 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484532 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484566 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/104ed972-6629-4e74-82b9-81e9dccec6bc-host-slash\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484572 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484569 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-dbus\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484593 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/234e4e76-252c-491b-b14f-6b9a7e2c36b9-tmp-dir\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484618 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-os-release\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.484690 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484678 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484725 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/104ed972-6629-4e74-82b9-81e9dccec6bc-iptables-alerter-script\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484737 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e85ff609-4afc-446a-83cd-77f70781dd24-os-release\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484750 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-registration-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.484802 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484831 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-device-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.484859 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret podName:ab2a9f80-37fa-4aa6-bc80-1914f1967e78 nodeName:}" failed. No retries permitted until 2026-06-02 01:22:54.984840201 +0000 UTC m=+3.189300465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret") pod "global-pull-secret-syncer-4vq4w" (UID: "ab2a9f80-37fa-4aa6-bc80-1914f1967e78") : object "kube-system"/"original-pull-secret" not registered Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484881 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-device-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484806 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-registration-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484950 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/234e4e76-252c-491b-b14f-6b9a7e2c36b9-tmp-dir\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.484894 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/234e4e76-252c-491b-b14f-6b9a7e2c36b9-hosts-file\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485006 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/234e4e76-252c-491b-b14f-6b9a7e2c36b9-hosts-file\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485007 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-socket-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485056 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-etc-selinux\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485082 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nc9bq\" (UniqueName: \"kubernetes.io/projected/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-kube-api-access-nc9bq\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485128 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-socket-dir\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.485237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485129 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.486017 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485168 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k464m\" (UniqueName: \"kubernetes.io/projected/e85ff609-4afc-446a-83cd-77f70781dd24-kube-api-access-k464m\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.486017 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485204 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.486017 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485586 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-etc-selinux\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.486017 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.485745 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/104ed972-6629-4e74-82b9-81e9dccec6bc-iptables-alerter-script\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.486193 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.486149 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e85ff609-4afc-446a-83cd-77f70781dd24-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.496315 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.496260 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ctxc\" (UniqueName: \"kubernetes.io/projected/104ed972-6629-4e74-82b9-81e9dccec6bc-kube-api-access-9ctxc\") pod \"iptables-alerter-shd9s\" (UID: \"104ed972-6629-4e74-82b9-81e9dccec6bc\") " pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.496315 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.496266 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc9bq\" (UniqueName: \"kubernetes.io/projected/f4eacf34-a661-4c6f-b8db-c5a42fd7aab1-kube-api-access-nc9bq\") pod \"aws-ebs-csi-driver-node-sgv6z\" (UID: \"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.496482 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.496381 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k464m\" (UniqueName: \"kubernetes.io/projected/e85ff609-4afc-446a-83cd-77f70781dd24-kube-api-access-k464m\") pod \"multus-additional-cni-plugins-7tmcl\" (UID: \"e85ff609-4afc-446a-83cd-77f70781dd24\") " pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.496602 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.496581 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr8jh\" (UniqueName: \"kubernetes.io/projected/234e4e76-252c-491b-b14f-6b9a7e2c36b9-kube-api-access-nr8jh\") pod \"node-resolver-wn6w8\" (UID: \"234e4e76-252c-491b-b14f-6b9a7e2c36b9\") " pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.573469 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.573438 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:22:54.580108 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.580088 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8lt9m" Jun 02 01:22:54.589897 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.589859 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:22:54.595452 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.595433 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hbngb" Jun 02 01:22:54.603036 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.603019 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-672sz" Jun 02 01:22:54.610668 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.610639 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" Jun 02 01:22:54.619216 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.619170 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-shd9s" Jun 02 01:22:54.625797 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.625782 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" Jun 02 01:22:54.632279 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.632259 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wn6w8" Jun 02 01:22:54.888972 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.888880 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:54.888972 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.888911 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:54.889182 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.889059 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:22:54.889182 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.889070 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 01:22:54.889182 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.889085 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 01:22:54.889182 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.889093 2564 projected.go:194] Error preparing data for projected volume kube-api-access-pxnvl for pod openshift-network-diagnostics/network-check-target-f7cf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:22:54.889182 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.889125 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs podName:cf148b2d-3a22-4a94-98f3-5dcc28eebd87 nodeName:}" failed. No retries permitted until 2026-06-02 01:22:55.889107803 +0000 UTC m=+4.093568062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs") pod "network-metrics-daemon-zvkvb" (UID: "cf148b2d-3a22-4a94-98f3-5dcc28eebd87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:22:54.889182 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.889147 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl podName:16ba9005-eca6-42a7-96ab-800c12788d7b nodeName:}" failed. No retries permitted until 2026-06-02 01:22:55.889136359 +0000 UTC m=+4.093596619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxnvl" (UniqueName: "kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl") pod "network-check-target-f7cf5" (UID: "16ba9005-eca6-42a7-96ab-800c12788d7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:22:54.989903 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:54.989872 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:54.990057 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.990010 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 01:22:54.990132 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:54.990083 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret podName:ab2a9f80-37fa-4aa6-bc80-1914f1967e78 nodeName:}" failed. No retries permitted until 2026-06-02 01:22:55.990062452 +0000 UTC m=+4.194522723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret") pod "global-pull-secret-syncer-4vq4w" (UID: "ab2a9f80-37fa-4aa6-bc80-1914f1967e78") : object "kube-system"/"original-pull-secret" not registered Jun 02 01:22:55.082343 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.082118 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b5b39e_4c8a_48ed_beae_191f194d012b.slice/crio-a16cb5507de072056f2ebd79861cf0541f3531afeeb260c8c8689d2378de54f9 WatchSource:0}: Error finding container a16cb5507de072056f2ebd79861cf0541f3531afeeb260c8c8689d2378de54f9: Status 404 returned error can't find the container with id a16cb5507de072056f2ebd79861cf0541f3531afeeb260c8c8689d2378de54f9 Jun 02 01:22:55.084475 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.084419 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4eacf34_a661_4c6f_b8db_c5a42fd7aab1.slice/crio-a06a2701c2413dccf481ca8802bdca9b0d60bf0bc81df2a13f0742b63d3fb325 WatchSource:0}: Error finding container a06a2701c2413dccf481ca8802bdca9b0d60bf0bc81df2a13f0742b63d3fb325: Status 404 returned error can't find the container with id a06a2701c2413dccf481ca8802bdca9b0d60bf0bc81df2a13f0742b63d3fb325 Jun 02 01:22:55.088744 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.088717 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85ff609_4afc_446a_83cd_77f70781dd24.slice/crio-4b64af2bfdf5856b161a75a1e5ce29a1601f4fb7ea284c9a734d8067c463d17d WatchSource:0}: Error finding container 4b64af2bfdf5856b161a75a1e5ce29a1601f4fb7ea284c9a734d8067c463d17d: Status 404 returned error can't find the container with id 4b64af2bfdf5856b161a75a1e5ce29a1601f4fb7ea284c9a734d8067c463d17d Jun 02 01:22:55.089562 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.089538 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962fbf55_761d_4399_9cd3_6d3cfb488044.slice/crio-6e471e7b1b9f2fadd29875c6b40bd77c10fdac7d47c278088478e93f149ae364 WatchSource:0}: Error finding container 6e471e7b1b9f2fadd29875c6b40bd77c10fdac7d47c278088478e93f149ae364: Status 404 returned error can't find the container with id 6e471e7b1b9f2fadd29875c6b40bd77c10fdac7d47c278088478e93f149ae364 Jun 02 01:22:55.091014 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.090985 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda415cede_b0c5_4300_8f99_22397439fe3f.slice/crio-f01dbccb55d4c78a3c4746dccfaa8b51a7da7e5ca2eae8180a5f706b20cbac0d WatchSource:0}: Error finding container f01dbccb55d4c78a3c4746dccfaa8b51a7da7e5ca2eae8180a5f706b20cbac0d: Status 404 returned error can't find the container with id f01dbccb55d4c78a3c4746dccfaa8b51a7da7e5ca2eae8180a5f706b20cbac0d Jun 02 01:22:55.091326 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.091293 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa180230_fe8b_48c0_be17_f47f8d9be3bc.slice/crio-655ba096b76e016af66cfaf2d9f6c91d007d157b1d8d36c350829f706cbd2f7e WatchSource:0}: Error finding container 655ba096b76e016af66cfaf2d9f6c91d007d157b1d8d36c350829f706cbd2f7e: Status 404 returned error can't find the container with id 655ba096b76e016af66cfaf2d9f6c91d007d157b1d8d36c350829f706cbd2f7e Jun 02 01:22:55.092361 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.092267 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401e1384_3579_4929_86af_eb642d871287.slice/crio-3d11bd0a5c6347ea5c316620543fa26c91cdc67f807392da09f6921bce36eba3 WatchSource:0}: Error finding container 3d11bd0a5c6347ea5c316620543fa26c91cdc67f807392da09f6921bce36eba3: Status 404 returned error can't find the container with id 3d11bd0a5c6347ea5c316620543fa26c91cdc67f807392da09f6921bce36eba3 Jun 02 01:22:55.093295 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.093274 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234e4e76_252c_491b_b14f_6b9a7e2c36b9.slice/crio-cbe839d5d6c11032b172cf463cb4d5fc8bfc4cdbabf05f4cefd73fd19643c529 WatchSource:0}: Error finding container cbe839d5d6c11032b172cf463cb4d5fc8bfc4cdbabf05f4cefd73fd19643c529: Status 404 returned error can't find the container with id cbe839d5d6c11032b172cf463cb4d5fc8bfc4cdbabf05f4cefd73fd19643c529 Jun 02 01:22:55.094092 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:22:55.093826 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod104ed972_6629_4e74_82b9_81e9dccec6bc.slice/crio-fa65e288d7a64b3fc5d7e5abb62bef0ffb7d8d70c44bad25021a662b15178a78 WatchSource:0}: Error finding container fa65e288d7a64b3fc5d7e5abb62bef0ffb7d8d70c44bad25021a662b15178a78: Status 404 returned error can't find the container with id fa65e288d7a64b3fc5d7e5abb62bef0ffb7d8d70c44bad25021a662b15178a78 Jun 02 01:22:55.307754 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.307713 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 01:17:53 +0000 UTC" deadline="2028-01-20 16:17:31.631153809 +0000 UTC" Jun 02 01:22:55.307754 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.307750 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14342h54m36.323407824s" Jun 02 01:22:55.373423 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.373403 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:55.373533 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.373516 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:22:55.373585 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.373527 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:55.373635 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.373618 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:22:55.380848 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.380826 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hbngb" event={"ID":"aa180230-fe8b-48c0-be17-f47f8d9be3bc","Type":"ContainerStarted","Data":"655ba096b76e016af66cfaf2d9f6c91d007d157b1d8d36c350829f706cbd2f7e"} Jun 02 01:22:55.381719 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.381697 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" event={"ID":"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1","Type":"ContainerStarted","Data":"a06a2701c2413dccf481ca8802bdca9b0d60bf0bc81df2a13f0742b63d3fb325"} Jun 02 01:22:55.383024 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.383005 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" event={"ID":"645034bd148c3fd0460f6af9403bc9a0","Type":"ContainerStarted","Data":"cf968a4e41cb5d2d548e641236bf76b219378952254005ec708cdfa778227a65"} Jun 02 01:22:55.384056 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.384037 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" event={"ID":"f81178f6a81c011305b0e997a8ecfc74","Type":"ContainerStarted","Data":"b95f0f377d1fecddeda04c6b994058a31907cb41001bce8596801a56f89a89d7"} Jun 02 01:22:55.384928 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.384911 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wn6w8" event={"ID":"234e4e76-252c-491b-b14f-6b9a7e2c36b9","Type":"ContainerStarted","Data":"cbe839d5d6c11032b172cf463cb4d5fc8bfc4cdbabf05f4cefd73fd19643c529"} Jun 02 01:22:55.385896 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.385871 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-672sz" event={"ID":"401e1384-3579-4929-86af-eb642d871287","Type":"ContainerStarted","Data":"3d11bd0a5c6347ea5c316620543fa26c91cdc67f807392da09f6921bce36eba3"} Jun 02 01:22:55.386806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.386789 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8lt9m" event={"ID":"a415cede-b0c5-4300-8f99-22397439fe3f","Type":"ContainerStarted","Data":"f01dbccb55d4c78a3c4746dccfaa8b51a7da7e5ca2eae8180a5f706b20cbac0d"} Jun 02 01:22:55.387699 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.387676 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"6e471e7b1b9f2fadd29875c6b40bd77c10fdac7d47c278088478e93f149ae364"} Jun 02 01:22:55.388555 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.388525 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" event={"ID":"e85ff609-4afc-446a-83cd-77f70781dd24","Type":"ContainerStarted","Data":"4b64af2bfdf5856b161a75a1e5ce29a1601f4fb7ea284c9a734d8067c463d17d"} Jun 02 01:22:55.389497 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.389477 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xkf6n" event={"ID":"52b5b39e-4c8a-48ed-beae-191f194d012b","Type":"ContainerStarted","Data":"a16cb5507de072056f2ebd79861cf0541f3531afeeb260c8c8689d2378de54f9"} Jun 02 01:22:55.390425 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.390406 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-shd9s" event={"ID":"104ed972-6629-4e74-82b9-81e9dccec6bc","Type":"ContainerStarted","Data":"fa65e288d7a64b3fc5d7e5abb62bef0ffb7d8d70c44bad25021a662b15178a78"} Jun 02 01:22:55.415390 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.415346 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-218.ec2.internal" podStartSLOduration=2.415334352 podStartE2EDuration="2.415334352s" podCreationTimestamp="2026-06-02 01:22:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 01:22:55.398291303 +0000 UTC m=+3.602751587" watchObservedRunningTime="2026-06-02 01:22:55.415334352 +0000 UTC m=+3.619794636" Jun 02 01:22:55.896989 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.896836 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:55.896989 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.896890 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:55.897238 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.897015 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:22:55.897238 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.897067 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 01:22:55.897238 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.897089 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 01:22:55.897238 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.897099 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs podName:cf148b2d-3a22-4a94-98f3-5dcc28eebd87 nodeName:}" failed. No retries permitted until 2026-06-02 01:22:57.897078742 +0000 UTC m=+6.101539005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs") pod "network-metrics-daemon-zvkvb" (UID: "cf148b2d-3a22-4a94-98f3-5dcc28eebd87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:22:55.897238 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.897102 2564 projected.go:194] Error preparing data for projected volume kube-api-access-pxnvl for pod openshift-network-diagnostics/network-check-target-f7cf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:22:55.897238 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.897143 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl podName:16ba9005-eca6-42a7-96ab-800c12788d7b nodeName:}" failed. No retries permitted until 2026-06-02 01:22:57.897133503 +0000 UTC m=+6.101593770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxnvl" (UniqueName: "kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl") pod "network-check-target-f7cf5" (UID: "16ba9005-eca6-42a7-96ab-800c12788d7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:22:55.998128 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:55.998091 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:55.998320 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.998302 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 01:22:55.998384 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:55.998377 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret podName:ab2a9f80-37fa-4aa6-bc80-1914f1967e78 nodeName:}" failed. No retries permitted until 2026-06-02 01:22:57.998356107 +0000 UTC m=+6.202816371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret") pod "global-pull-secret-syncer-4vq4w" (UID: "ab2a9f80-37fa-4aa6-bc80-1914f1967e78") : object "kube-system"/"original-pull-secret" not registered Jun 02 01:22:56.190055 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:56.189978 2564 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 01:22:56.374830 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:56.374799 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:56.375293 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:56.374927 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:22:56.409192 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:56.408020 2564 generic.go:358] "Generic (PLEG): container finished" podID="f81178f6a81c011305b0e997a8ecfc74" containerID="b95f0f377d1fecddeda04c6b994058a31907cb41001bce8596801a56f89a89d7" exitCode=0 Jun 02 01:22:56.409192 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:56.409019 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" event={"ID":"f81178f6a81c011305b0e997a8ecfc74","Type":"ContainerDied","Data":"b95f0f377d1fecddeda04c6b994058a31907cb41001bce8596801a56f89a89d7"} Jun 02 01:22:57.373662 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:57.372923 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:57.373662 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:57.373063 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:22:57.373662 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:57.373170 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:57.373662 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:57.373262 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:22:57.422722 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:57.422685 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" event={"ID":"f81178f6a81c011305b0e997a8ecfc74","Type":"ContainerStarted","Data":"091866114b22f51882326779d38a6e4a420008e7f00289aa138fed1e35acef2a"} Jun 02 01:22:57.442222 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:57.442160 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-218.ec2.internal" podStartSLOduration=4.442143604 podStartE2EDuration="4.442143604s" podCreationTimestamp="2026-06-02 01:22:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 01:22:57.437278039 +0000 UTC m=+5.641738323" watchObservedRunningTime="2026-06-02 01:22:57.442143604 +0000 UTC m=+5.646603887" Jun 02 01:22:57.918241 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:57.918205 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:57.918433 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:57.918254 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:57.918433 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:57.918408 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 01:22:57.918433 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:57.918427 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 01:22:57.918576 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:57.918442 2564 projected.go:194] Error preparing data for projected volume kube-api-access-pxnvl for pod openshift-network-diagnostics/network-check-target-f7cf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:22:57.918576 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:57.918503 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl podName:16ba9005-eca6-42a7-96ab-800c12788d7b nodeName:}" failed. No retries permitted until 2026-06-02 01:23:01.918483057 +0000 UTC m=+10.122943332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxnvl" (UniqueName: "kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl") pod "network-check-target-f7cf5" (UID: "16ba9005-eca6-42a7-96ab-800c12788d7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:22:57.918968 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:57.918948 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:22:57.919061 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:57.919005 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs podName:cf148b2d-3a22-4a94-98f3-5dcc28eebd87 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:01.918988719 +0000 UTC m=+10.123448994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs") pod "network-metrics-daemon-zvkvb" (UID: "cf148b2d-3a22-4a94-98f3-5dcc28eebd87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:22:58.018968 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:58.018930 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:58.019147 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:58.019084 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 01:22:58.019147 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:58.019145 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret podName:ab2a9f80-37fa-4aa6-bc80-1914f1967e78 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:02.019128416 +0000 UTC m=+10.223588689 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret") pod "global-pull-secret-syncer-4vq4w" (UID: "ab2a9f80-37fa-4aa6-bc80-1914f1967e78") : object "kube-system"/"original-pull-secret" not registered Jun 02 01:22:58.373164 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:58.373134 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:22:58.373331 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:58.373231 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:22:59.373027 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:59.372994 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:22:59.373505 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:59.373110 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:22:59.373505 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:22:59.373169 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:22:59.373505 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:22:59.373235 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:00.373819 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:00.373780 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:00.374248 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:00.373927 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:01.373587 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:01.373551 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:01.373792 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:01.373705 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:01.374099 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:01.373558 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:01.374099 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:01.374062 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:01.952460 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:01.952420 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:01.952646 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:01.952471 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:01.952737 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:01.952664 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 01:23:01.952737 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:01.952690 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 01:23:01.952737 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:01.952704 2564 projected.go:194] Error preparing data for projected volume kube-api-access-pxnvl for pod openshift-network-diagnostics/network-check-target-f7cf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:23:01.952737 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:01.952706 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:23:01.952936 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:01.952759 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl podName:16ba9005-eca6-42a7-96ab-800c12788d7b nodeName:}" failed. No retries permitted until 2026-06-02 01:23:09.952740551 +0000 UTC m=+18.157200825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxnvl" (UniqueName: "kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl") pod "network-check-target-f7cf5" (UID: "16ba9005-eca6-42a7-96ab-800c12788d7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:23:01.952936 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:01.952779 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs podName:cf148b2d-3a22-4a94-98f3-5dcc28eebd87 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:09.952769329 +0000 UTC m=+18.157229589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs") pod "network-metrics-daemon-zvkvb" (UID: "cf148b2d-3a22-4a94-98f3-5dcc28eebd87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:23:02.053294 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:02.053258 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:02.053436 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:02.053386 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 01:23:02.053505 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:02.053451 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret podName:ab2a9f80-37fa-4aa6-bc80-1914f1967e78 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:10.053433325 +0000 UTC m=+18.257893584 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret") pod "global-pull-secret-syncer-4vq4w" (UID: "ab2a9f80-37fa-4aa6-bc80-1914f1967e78") : object "kube-system"/"original-pull-secret" not registered Jun 02 01:23:02.374217 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:02.373730 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:02.374217 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:02.373845 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:03.373312 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:03.373282 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:03.373312 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:03.373298 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:03.373515 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:03.373406 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:03.373515 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:03.373492 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:04.372985 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:04.372944 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:04.373468 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:04.373068 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:05.373532 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:05.373506 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:05.373532 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:05.373522 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:05.373972 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:05.373608 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:05.373972 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:05.373716 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:06.373802 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:06.373769 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:06.374229 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:06.373898 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:07.373480 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:07.373449 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:07.373694 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:07.373449 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:07.373694 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:07.373564 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:07.373694 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:07.373648 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:08.372831 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:08.372796 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:08.373215 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:08.372962 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:09.373724 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:09.373676 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:09.374332 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:09.373684 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:09.374332 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:09.373797 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:09.374332 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:09.373881 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:10.014072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:10.014028 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:10.014072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:10.014078 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:10.014294 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.014213 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:23:10.014294 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.014226 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 01:23:10.014294 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.014240 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 01:23:10.014294 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.014252 2564 projected.go:194] Error preparing data for projected volume kube-api-access-pxnvl for pod openshift-network-diagnostics/network-check-target-f7cf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:23:10.014294 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.014283 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs podName:cf148b2d-3a22-4a94-98f3-5dcc28eebd87 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:26.014262701 +0000 UTC m=+34.218722978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs") pod "network-metrics-daemon-zvkvb" (UID: "cf148b2d-3a22-4a94-98f3-5dcc28eebd87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:23:10.014450 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.014300 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl podName:16ba9005-eca6-42a7-96ab-800c12788d7b nodeName:}" failed. No retries permitted until 2026-06-02 01:23:26.01429332 +0000 UTC m=+34.218753580 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxnvl" (UniqueName: "kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl") pod "network-check-target-f7cf5" (UID: "16ba9005-eca6-42a7-96ab-800c12788d7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:23:10.114754 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:10.114721 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:10.114925 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.114858 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 01:23:10.114925 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.114923 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret podName:ab2a9f80-37fa-4aa6-bc80-1914f1967e78 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:26.114909321 +0000 UTC m=+34.319369586 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret") pod "global-pull-secret-syncer-4vq4w" (UID: "ab2a9f80-37fa-4aa6-bc80-1914f1967e78") : object "kube-system"/"original-pull-secret" not registered Jun 02 01:23:10.373835 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:10.373766 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:10.374286 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:10.373895 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:11.373372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:11.373337 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:11.373533 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:11.373345 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:11.373533 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:11.373443 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:11.373666 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:11.373548 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:12.373727 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:12.373703 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:12.374135 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:12.373800 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:13.372997 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:13.372959 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:13.372997 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:13.372996 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:13.373160 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:13.373075 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:13.373204 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:13.373160 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:14.373938 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.373751 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:14.374688 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:14.374025 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:14.450056 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.450017 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wn6w8" event={"ID":"234e4e76-252c-491b-b14f-6b9a7e2c36b9","Type":"ContainerStarted","Data":"20882bd7d4b939eb9eb6ebf3bd4fb096178426223a4752498bdb13a14992e98b"} Jun 02 01:23:14.453317 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.453281 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-672sz" event={"ID":"401e1384-3579-4929-86af-eb642d871287","Type":"ContainerStarted","Data":"5f204ef3b38512decf11be7a6114cc0d9bd0371ea6196d4266a4adea3443e4f9"} Jun 02 01:23:14.457361 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.457335 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8lt9m" event={"ID":"a415cede-b0c5-4300-8f99-22397439fe3f","Type":"ContainerStarted","Data":"c2cdb3c563611af0bc32501e0458359448e8d0622fa3dae985a23aa367b60a8a"} Jun 02 01:23:14.460092 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.460072 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"8ce0f9c6c5938c8dbfdf039fd055925183db2fa4c3beefc376da5dcc0385cf24"} Jun 02 01:23:14.460197 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.460097 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"23dfc9d6c1da5988e00bc40a84f5bb4ab4d45c5dfd3a4f2e3a53f4021ef760dd"} Jun 02 01:23:14.460197 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.460111 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"28f5bdc766eb2f99792d309d6eeba484365281579343d78bae409d3ebc76e55a"} Jun 02 01:23:14.460197 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.460123 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"8a6117cf35a506636e97094a0926c2c763f6bbd81a078c9652ea1312454a55ae"} Jun 02 01:23:14.460197 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.460136 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"c8a3715cc2d2af01252590ffa2e38497219996940780b8048d31729bb285a168"} Jun 02 01:23:14.460197 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.460147 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"1f7d25f35c011df40bf4ca35f07309b30870fb2ffe3c5842ceaaf6f4ffb83510"} Jun 02 01:23:14.461442 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.461422 2564 generic.go:358] "Generic (PLEG): container finished" podID="e85ff609-4afc-446a-83cd-77f70781dd24" containerID="60232487098f4d1e0697d1117b46ef782010b14ec35524b30a29ea73a047864e" exitCode=0 Jun 02 01:23:14.461540 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.461486 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" event={"ID":"e85ff609-4afc-446a-83cd-77f70781dd24","Type":"ContainerDied","Data":"60232487098f4d1e0697d1117b46ef782010b14ec35524b30a29ea73a047864e"} Jun 02 01:23:14.463011 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.462982 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xkf6n" event={"ID":"52b5b39e-4c8a-48ed-beae-191f194d012b","Type":"ContainerStarted","Data":"fc2025fed8f4505368f6fc701eab584c28d6560e7491339476196712c2a22b7c"} Jun 02 01:23:14.464450 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.464425 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hbngb" event={"ID":"aa180230-fe8b-48c0-be17-f47f8d9be3bc","Type":"ContainerStarted","Data":"009b713bcbcf675cbd84bcf991b81a4765ea2eddd5086a43c463d6cc3bc72a6d"} Jun 02 01:23:14.465861 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.465839 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" event={"ID":"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1","Type":"ContainerStarted","Data":"41e39cf8e4502801fab038e16cf06b58bf2178d98e52e4bb6ee2e96bba5bb324"} Jun 02 01:23:14.466164 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.466128 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-wn6w8" podStartSLOduration=8.041273671 podStartE2EDuration="22.466115781s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.095161433 +0000 UTC m=+3.299621699" lastFinishedPulling="2026-06-02 01:23:09.520003535 +0000 UTC m=+17.724463809" observedRunningTime="2026-06-02 01:23:14.46563869 +0000 UTC m=+22.670099030" watchObservedRunningTime="2026-06-02 01:23:14.466115781 +0000 UTC m=+22.670576064" Jun 02 01:23:14.480604 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.480559 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xkf6n" podStartSLOduration=8.045274909 podStartE2EDuration="22.48054383s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.08472054 +0000 UTC m=+3.289180814" lastFinishedPulling="2026-06-02 01:23:09.519989461 +0000 UTC m=+17.724449735" observedRunningTime="2026-06-02 01:23:14.479784231 +0000 UTC m=+22.684244513" watchObservedRunningTime="2026-06-02 01:23:14.48054383 +0000 UTC m=+22.685004113" Jun 02 01:23:14.495434 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.495401 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hbngb" podStartSLOduration=4.150159592 podStartE2EDuration="22.495388185s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.093489962 +0000 UTC m=+3.297950236" lastFinishedPulling="2026-06-02 01:23:13.438718555 +0000 UTC m=+21.643178829" observedRunningTime="2026-06-02 01:23:14.495255549 +0000 UTC m=+22.699715822" watchObservedRunningTime="2026-06-02 01:23:14.495388185 +0000 UTC m=+22.699848471" Jun 02 01:23:14.531984 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.531930 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8lt9m" podStartSLOduration=8.104678455 podStartE2EDuration="22.531916969s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.092739057 +0000 UTC m=+3.297199320" lastFinishedPulling="2026-06-02 01:23:09.519977559 +0000 UTC m=+17.724437834" observedRunningTime="2026-06-02 01:23:14.531753029 +0000 UTC m=+22.736213312" watchObservedRunningTime="2026-06-02 01:23:14.531916969 +0000 UTC m=+22.736377250" Jun 02 01:23:14.547571 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.547529 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-672sz" podStartSLOduration=4.174238435 podStartE2EDuration="22.547516712s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.094107181 +0000 UTC m=+3.298567445" lastFinishedPulling="2026-06-02 01:23:13.467385462 +0000 UTC m=+21.671845722" observedRunningTime="2026-06-02 01:23:14.546806397 +0000 UTC m=+22.751266678" watchObservedRunningTime="2026-06-02 01:23:14.547516712 +0000 UTC m=+22.751976995" Jun 02 01:23:14.736088 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:14.736068 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 01:23:15.337173 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.336982 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T01:23:14.736084075Z","UUID":"c1d73b84-ad34-4f7e-b717-acc4bb5569bd","Handler":null,"Name":"","Endpoint":""} Jun 02 01:23:15.338745 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.338720 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 01:23:15.338745 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.338748 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 01:23:15.373015 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.372987 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:15.373171 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:15.373109 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:15.373439 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.372990 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:15.373439 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:15.373393 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:15.469448 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.469410 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" event={"ID":"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1","Type":"ContainerStarted","Data":"7edec9b6334c26a3ee377bc2828e0a13a88cc663aacdf10c55fe5c16773a61dd"} Jun 02 01:23:15.470996 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.470973 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-shd9s" event={"ID":"104ed972-6629-4e74-82b9-81e9dccec6bc","Type":"ContainerStarted","Data":"8c074279943c33a46a366ba7e93a79365820196e0d9d640d9b391002c7769b1a"} Jun 02 01:23:15.484707 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.484645 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-shd9s" podStartSLOduration=5.143831987 podStartE2EDuration="23.484633184s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.096095172 +0000 UTC m=+3.300555439" lastFinishedPulling="2026-06-02 01:23:13.436896358 +0000 UTC m=+21.641356636" observedRunningTime="2026-06-02 01:23:15.484399905 +0000 UTC m=+23.688860187" watchObservedRunningTime="2026-06-02 01:23:15.484633184 +0000 UTC m=+23.689093483" Jun 02 01:23:15.770073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.770048 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:23:15.770594 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:15.770580 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:23:16.373020 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:16.372784 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:16.373154 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:16.373132 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:16.476520 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:16.476487 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"8203eef6a3c95811ab1d08df85881b0b1ee98f10ad74acc63a14dd0b1c3c5386"} Jun 02 01:23:16.478180 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:16.478153 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" event={"ID":"f4eacf34-a661-4c6f-b8db-c5a42fd7aab1","Type":"ContainerStarted","Data":"b758e7935eb4d231820eafca46c19aa99d2904fb187bae4221578a91f098a4f8"} Jun 02 01:23:16.478731 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:16.478715 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:23:16.479172 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:16.479157 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xkf6n" Jun 02 01:23:16.495243 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:16.495180 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sgv6z" podStartSLOduration=4.021603959 podStartE2EDuration="24.495170684s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.086978657 +0000 UTC m=+3.291438916" lastFinishedPulling="2026-06-02 01:23:15.560545381 +0000 UTC m=+23.765005641" observedRunningTime="2026-06-02 01:23:16.495150583 +0000 UTC m=+24.699610865" watchObservedRunningTime="2026-06-02 01:23:16.495170684 +0000 UTC m=+24.699630966" Jun 02 01:23:17.373062 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:17.373027 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:17.373062 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:17.373051 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:17.373332 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:17.373135 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:17.373332 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:17.373281 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:18.372950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:18.372929 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:18.373521 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:18.373042 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:18.485139 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:18.484968 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" event={"ID":"962fbf55-761d-4399-9cd3-6d3cfb488044","Type":"ContainerStarted","Data":"bd588702f1126a1cc95a953cffc6cf173fd6d00937dcc853d576fa5ad4577c29"} Jun 02 01:23:18.519445 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:18.519251 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" podStartSLOduration=7.977070013 podStartE2EDuration="26.519233198s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.091892603 +0000 UTC m=+3.296352867" lastFinishedPulling="2026-06-02 01:23:13.634055784 +0000 UTC m=+21.838516052" observedRunningTime="2026-06-02 01:23:18.5183347 +0000 UTC m=+26.722795004" watchObservedRunningTime="2026-06-02 01:23:18.519233198 +0000 UTC m=+26.723693481" Jun 02 01:23:19.373325 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:19.373294 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:19.373814 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:19.373294 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:19.373814 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:19.373417 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:19.373814 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:19.373517 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:19.487559 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:19.487525 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:23:19.487559 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:19.487568 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:23:19.487827 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:19.487583 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:23:19.504891 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:19.504863 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:23:19.505043 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:19.504961 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:23:20.355372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:20.355342 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4vq4w"] Jun 02 01:23:20.355513 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:20.355459 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:20.355561 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:20.355540 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:20.358283 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:20.358257 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zvkvb"] Jun 02 01:23:20.358397 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:20.358346 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:20.358477 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:20.358454 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:20.358763 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:20.358735 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f7cf5"] Jun 02 01:23:20.358863 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:20.358803 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:20.358902 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:20.358872 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:21.373195 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:21.373021 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:21.373530 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:21.373266 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:21.492543 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:21.492505 2564 generic.go:358] "Generic (PLEG): container finished" podID="e85ff609-4afc-446a-83cd-77f70781dd24" containerID="7c2b30b157dceaea791d68579b6f82ca3e1dfdc5383883a8b35c0b840323abba" exitCode=0 Jun 02 01:23:21.494237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:21.494163 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" event={"ID":"e85ff609-4afc-446a-83cd-77f70781dd24","Type":"ContainerDied","Data":"7c2b30b157dceaea791d68579b6f82ca3e1dfdc5383883a8b35c0b840323abba"} Jun 02 01:23:22.373957 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:22.373930 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:22.374323 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:22.374033 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:22.374323 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:22.374059 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:22.374323 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:22.374165 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:23.373309 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:23.373287 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:23.373429 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:23.373377 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:23.498095 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:23.498062 2564 generic.go:358] "Generic (PLEG): container finished" podID="e85ff609-4afc-446a-83cd-77f70781dd24" containerID="db0c6b8c5274911b0b26c7423a187fd8f80840670731a836a21cfc160d5a4f43" exitCode=0 Jun 02 01:23:23.498095 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:23.498097 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" event={"ID":"e85ff609-4afc-446a-83cd-77f70781dd24","Type":"ContainerDied","Data":"db0c6b8c5274911b0b26c7423a187fd8f80840670731a836a21cfc160d5a4f43"} Jun 02 01:23:24.373378 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:24.373350 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:24.373527 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:24.373361 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:24.373527 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:24.373442 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvkvb" podUID="cf148b2d-3a22-4a94-98f3-5dcc28eebd87" Jun 02 01:23:24.373613 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:24.373540 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7cf5" podUID="16ba9005-eca6-42a7-96ab-800c12788d7b" Jun 02 01:23:25.373627 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:25.373597 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:25.374169 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:25.373701 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vq4w" podUID="ab2a9f80-37fa-4aa6-bc80-1914f1967e78" Jun 02 01:23:25.502497 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:25.502466 2564 generic.go:358] "Generic (PLEG): container finished" podID="e85ff609-4afc-446a-83cd-77f70781dd24" containerID="6430970c08d51d8b34406980c0632200267f3898f684fb5ca70e1f03519858c3" exitCode=0 Jun 02 01:23:25.502619 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:25.502503 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" event={"ID":"e85ff609-4afc-446a-83cd-77f70781dd24","Type":"ContainerDied","Data":"6430970c08d51d8b34406980c0632200267f3898f684fb5ca70e1f03519858c3"} Jun 02 01:23:26.047550 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.047517 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:26.047550 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.047557 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:26.047793 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.047687 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:23:26.047793 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.047702 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 01:23:26.047793 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.047721 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 01:23:26.047793 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.047733 2564 projected.go:194] Error preparing data for projected volume kube-api-access-pxnvl for pod openshift-network-diagnostics/network-check-target-f7cf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:23:26.047793 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.047772 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs podName:cf148b2d-3a22-4a94-98f3-5dcc28eebd87 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:58.047750788 +0000 UTC m=+66.252211054 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs") pod "network-metrics-daemon-zvkvb" (UID: "cf148b2d-3a22-4a94-98f3-5dcc28eebd87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 01:23:26.047793 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.047792 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl podName:16ba9005-eca6-42a7-96ab-800c12788d7b nodeName:}" failed. No retries permitted until 2026-06-02 01:23:58.047783076 +0000 UTC m=+66.252243342 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxnvl" (UniqueName: "kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl") pod "network-check-target-f7cf5" (UID: "16ba9005-eca6-42a7-96ab-800c12788d7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 01:23:26.148010 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.147963 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:26.148178 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.148100 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 01:23:26.148178 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.148174 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret podName:ab2a9f80-37fa-4aa6-bc80-1914f1967e78 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:58.148157718 +0000 UTC m=+66.352617978 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret") pod "global-pull-secret-syncer-4vq4w" (UID: "ab2a9f80-37fa-4aa6-bc80-1914f1967e78") : object "kube-system"/"original-pull-secret" not registered Jun 02 01:23:26.151560 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.151530 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-218.ec2.internal" event="NodeReady" Jun 02 01:23:26.151801 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.151677 2564 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 01:23:26.196500 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.196471 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tfpsh"] Jun 02 01:23:26.198262 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.198241 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gtr8q"] Jun 02 01:23:26.198428 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.198413 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.201681 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.200832 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 01:23:26.201681 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.200986 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:26.201681 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.201605 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8tqnd\"" Jun 02 01:23:26.201878 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.201790 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 01:23:26.203496 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.203475 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 01:23:26.203612 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.203494 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 01:23:26.203612 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.203475 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xz25b\"" Jun 02 01:23:26.203805 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.203790 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 01:23:26.206849 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.206698 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gtr8q"] Jun 02 01:23:26.211506 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.211457 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tfpsh"] Jun 02 01:23:26.349143 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.349111 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.349143 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.349146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49vnx\" (UniqueName: \"kubernetes.io/projected/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-kube-api-access-49vnx\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.349379 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.349172 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sblfp\" (UniqueName: \"kubernetes.io/projected/99aff132-a1e2-4586-80cb-cee5ad8238bc-kube-api-access-sblfp\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:26.349379 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.349236 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-config-volume\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.349379 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.349292 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:26.349379 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.349361 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-tmp-dir\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.373289 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.373256 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:26.373451 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.373432 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:26.376298 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.376195 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 01:23:26.376298 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.376201 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jprs5\"" Jun 02 01:23:26.376852 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.376365 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 01:23:26.376852 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.376463 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tm89p\"" Jun 02 01:23:26.376852 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.376467 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 01:23:26.450357 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.450326 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:26.450357 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.450362 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-tmp-dir\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.450578 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.450498 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 01:23:26.450578 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.450563 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert podName:99aff132-a1e2-4586-80cb-cee5ad8238bc nodeName:}" failed. No retries permitted until 2026-06-02 01:23:26.950540832 +0000 UTC m=+35.155001092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert") pod "ingress-canary-gtr8q" (UID: "99aff132-a1e2-4586-80cb-cee5ad8238bc") : secret "canary-serving-cert" not found Jun 02 01:23:26.450702 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.450595 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.450702 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.450624 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-49vnx\" (UniqueName: \"kubernetes.io/projected/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-kube-api-access-49vnx\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.450702 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.450677 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-tmp-dir\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.450845 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.450723 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 01:23:26.450845 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.450770 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls podName:0a88a237-04e2-41a9-a1c5-dedfb5bc6df0 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:26.950754307 +0000 UTC m=+35.155214570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls") pod "dns-default-tfpsh" (UID: "0a88a237-04e2-41a9-a1c5-dedfb5bc6df0") : secret "dns-default-metrics-tls" not found Jun 02 01:23:26.450845 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.450790 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sblfp\" (UniqueName: \"kubernetes.io/projected/99aff132-a1e2-4586-80cb-cee5ad8238bc-kube-api-access-sblfp\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:26.450845 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.450815 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-config-volume\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.451255 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.451239 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-config-volume\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.462313 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.462244 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49vnx\" (UniqueName: \"kubernetes.io/projected/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-kube-api-access-49vnx\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.462482 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.462467 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sblfp\" (UniqueName: \"kubernetes.io/projected/99aff132-a1e2-4586-80cb-cee5ad8238bc-kube-api-access-sblfp\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:26.955609 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.955576 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:26.955809 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:26.955729 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:26.955809 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.955738 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 01:23:26.955899 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.955782 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 01:23:26.955899 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.955867 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert podName:99aff132-a1e2-4586-80cb-cee5ad8238bc nodeName:}" failed. No retries permitted until 2026-06-02 01:23:27.955847757 +0000 UTC m=+36.160308023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert") pod "ingress-canary-gtr8q" (UID: "99aff132-a1e2-4586-80cb-cee5ad8238bc") : secret "canary-serving-cert" not found Jun 02 01:23:26.955988 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:26.955909 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls podName:0a88a237-04e2-41a9-a1c5-dedfb5bc6df0 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:27.955891919 +0000 UTC m=+36.160352192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls") pod "dns-default-tfpsh" (UID: "0a88a237-04e2-41a9-a1c5-dedfb5bc6df0") : secret "dns-default-metrics-tls" not found Jun 02 01:23:27.372700 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:27.372674 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:27.375629 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:27.375605 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 01:23:27.963121 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:27.963081 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:27.963575 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:27.963163 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:27.963575 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:27.963234 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 01:23:27.963575 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:27.963269 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 01:23:27.963575 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:27.963310 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert podName:99aff132-a1e2-4586-80cb-cee5ad8238bc nodeName:}" failed. No retries permitted until 2026-06-02 01:23:29.963292306 +0000 UTC m=+38.167752583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert") pod "ingress-canary-gtr8q" (UID: "99aff132-a1e2-4586-80cb-cee5ad8238bc") : secret "canary-serving-cert" not found Jun 02 01:23:27.963575 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:27.963327 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls podName:0a88a237-04e2-41a9-a1c5-dedfb5bc6df0 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:29.963319844 +0000 UTC m=+38.167780103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls") pod "dns-default-tfpsh" (UID: "0a88a237-04e2-41a9-a1c5-dedfb5bc6df0") : secret "dns-default-metrics-tls" not found Jun 02 01:23:29.980963 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:29.980754 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:29.981380 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:29.981025 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:29.981380 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:29.980942 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 01:23:29.981380 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:29.981115 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert podName:99aff132-a1e2-4586-80cb-cee5ad8238bc nodeName:}" failed. No retries permitted until 2026-06-02 01:23:33.981092131 +0000 UTC m=+42.185552390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert") pod "ingress-canary-gtr8q" (UID: "99aff132-a1e2-4586-80cb-cee5ad8238bc") : secret "canary-serving-cert" not found Jun 02 01:23:29.981380 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:29.981138 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 01:23:29.981380 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:29.981176 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls podName:0a88a237-04e2-41a9-a1c5-dedfb5bc6df0 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:33.981165497 +0000 UTC m=+42.185625758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls") pod "dns-default-tfpsh" (UID: "0a88a237-04e2-41a9-a1c5-dedfb5bc6df0") : secret "dns-default-metrics-tls" not found Jun 02 01:23:31.515164 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:31.515132 2564 generic.go:358] "Generic (PLEG): container finished" podID="e85ff609-4afc-446a-83cd-77f70781dd24" containerID="2d234902b6168e9bed2058214461cf4bfa381e0e0e1eee9e30fcc2bd13172a5b" exitCode=0 Jun 02 01:23:31.515164 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:31.515177 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" event={"ID":"e85ff609-4afc-446a-83cd-77f70781dd24","Type":"ContainerDied","Data":"2d234902b6168e9bed2058214461cf4bfa381e0e0e1eee9e30fcc2bd13172a5b"} Jun 02 01:23:32.519027 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:32.518997 2564 generic.go:358] "Generic (PLEG): container finished" podID="e85ff609-4afc-446a-83cd-77f70781dd24" containerID="83a464e555146650b5377528cf773b58ef50ea822b847a1ce1eab13a101f085c" exitCode=0 Jun 02 01:23:32.519399 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:32.519042 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" event={"ID":"e85ff609-4afc-446a-83cd-77f70781dd24","Type":"ContainerDied","Data":"83a464e555146650b5377528cf773b58ef50ea822b847a1ce1eab13a101f085c"} Jun 02 01:23:33.523771 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:33.523734 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" event={"ID":"e85ff609-4afc-446a-83cd-77f70781dd24","Type":"ContainerStarted","Data":"829bbd1673d3d12734aa36c6a03b38dadb9fdb6d341f8968013bee299c53e77c"} Jun 02 01:23:33.546160 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:33.546119 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7tmcl" podStartSLOduration=5.807247158 podStartE2EDuration="41.5461001s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:22:55.090429939 +0000 UTC m=+3.294890205" lastFinishedPulling="2026-06-02 01:23:30.829282872 +0000 UTC m=+39.033743147" observedRunningTime="2026-06-02 01:23:33.545250604 +0000 UTC m=+41.749710886" watchObservedRunningTime="2026-06-02 01:23:33.5461001 +0000 UTC m=+41.750560381" Jun 02 01:23:34.009810 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:34.009773 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:34.009984 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:34.009836 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:34.009984 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:34.009920 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 01:23:34.010060 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:34.009987 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert podName:99aff132-a1e2-4586-80cb-cee5ad8238bc nodeName:}" failed. No retries permitted until 2026-06-02 01:23:42.009972444 +0000 UTC m=+50.214432704 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert") pod "ingress-canary-gtr8q" (UID: "99aff132-a1e2-4586-80cb-cee5ad8238bc") : secret "canary-serving-cert" not found Jun 02 01:23:34.010060 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:34.009925 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 01:23:34.010060 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:34.010055 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls podName:0a88a237-04e2-41a9-a1c5-dedfb5bc6df0 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:42.01004245 +0000 UTC m=+50.214502711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls") pod "dns-default-tfpsh" (UID: "0a88a237-04e2-41a9-a1c5-dedfb5bc6df0") : secret "dns-default-metrics-tls" not found Jun 02 01:23:42.064447 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:42.064400 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:42.064928 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:42.064481 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:42.064928 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:42.064548 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 01:23:42.064928 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:42.064569 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 01:23:42.064928 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:42.064611 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls podName:0a88a237-04e2-41a9-a1c5-dedfb5bc6df0 nodeName:}" failed. No retries permitted until 2026-06-02 01:23:58.064597437 +0000 UTC m=+66.269057697 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls") pod "dns-default-tfpsh" (UID: "0a88a237-04e2-41a9-a1c5-dedfb5bc6df0") : secret "dns-default-metrics-tls" not found Jun 02 01:23:42.064928 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:42.064624 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert podName:99aff132-a1e2-4586-80cb-cee5ad8238bc nodeName:}" failed. No retries permitted until 2026-06-02 01:23:58.064618085 +0000 UTC m=+66.269078345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert") pod "ingress-canary-gtr8q" (UID: "99aff132-a1e2-4586-80cb-cee5ad8238bc") : secret "canary-serving-cert" not found Jun 02 01:23:51.504910 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:51.504883 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qsmk7" Jun 02 01:23:58.081966 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.081926 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:23:58.082551 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.081982 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:23:58.082551 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.082017 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:23:58.082551 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.082035 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:58.082551 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:58.082088 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 01:23:58.082551 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:58.082147 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 01:23:58.082551 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:58.082168 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls podName:0a88a237-04e2-41a9-a1c5-dedfb5bc6df0 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:30.082145277 +0000 UTC m=+98.286605548 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls") pod "dns-default-tfpsh" (UID: "0a88a237-04e2-41a9-a1c5-dedfb5bc6df0") : secret "dns-default-metrics-tls" not found Jun 02 01:23:58.082551 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:58.082215 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert podName:99aff132-a1e2-4586-80cb-cee5ad8238bc nodeName:}" failed. No retries permitted until 2026-06-02 01:24:30.082199988 +0000 UTC m=+98.286660249 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert") pod "ingress-canary-gtr8q" (UID: "99aff132-a1e2-4586-80cb-cee5ad8238bc") : secret "canary-serving-cert" not found Jun 02 01:23:58.085045 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.085028 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 01:23:58.085124 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.085063 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 01:23:58.092325 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:58.092310 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 01:23:58.092389 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:23:58.092350 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs podName:cf148b2d-3a22-4a94-98f3-5dcc28eebd87 nodeName:}" failed. No retries permitted until 2026-06-02 01:25:02.092339132 +0000 UTC m=+130.296799391 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs") pod "network-metrics-daemon-zvkvb" (UID: "cf148b2d-3a22-4a94-98f3-5dcc28eebd87") : secret "metrics-daemon-secret" not found Jun 02 01:23:58.094363 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.094348 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 01:23:58.106123 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.106099 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxnvl\" (UniqueName: \"kubernetes.io/projected/16ba9005-eca6-42a7-96ab-800c12788d7b-kube-api-access-pxnvl\") pod \"network-check-target-f7cf5\" (UID: \"16ba9005-eca6-42a7-96ab-800c12788d7b\") " pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:58.183265 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.183242 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:58.185910 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.185895 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 01:23:58.191760 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.191745 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tm89p\"" Jun 02 01:23:58.195732 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.195715 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2a9f80-37fa-4aa6-bc80-1914f1967e78-original-pull-secret\") pod \"global-pull-secret-syncer-4vq4w\" (UID: \"ab2a9f80-37fa-4aa6-bc80-1914f1967e78\") " pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:58.200681 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.200661 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:23:58.284115 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.284077 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vq4w" Jun 02 01:23:58.323913 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.323866 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f7cf5"] Jun 02 01:23:58.327729 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:23:58.327694 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ba9005_eca6_42a7_96ab_800c12788d7b.slice/crio-8fe97a4ad9f57ba6a565b449b96bef4a18824479c504c94547ac22c5e271bb89 WatchSource:0}: Error finding container 8fe97a4ad9f57ba6a565b449b96bef4a18824479c504c94547ac22c5e271bb89: Status 404 returned error can't find the container with id 8fe97a4ad9f57ba6a565b449b96bef4a18824479c504c94547ac22c5e271bb89 Jun 02 01:23:58.419349 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.419313 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4vq4w"] Jun 02 01:23:58.424840 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:23:58.424813 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab2a9f80_37fa_4aa6_bc80_1914f1967e78.slice/crio-e547130ed3c6a3c2b4a430e5bfc1db84ec78249f1aff89d4b3fd822184cd1487 WatchSource:0}: Error finding container e547130ed3c6a3c2b4a430e5bfc1db84ec78249f1aff89d4b3fd822184cd1487: Status 404 returned error can't find the container with id e547130ed3c6a3c2b4a430e5bfc1db84ec78249f1aff89d4b3fd822184cd1487 Jun 02 01:23:58.570833 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.570801 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4vq4w" event={"ID":"ab2a9f80-37fa-4aa6-bc80-1914f1967e78","Type":"ContainerStarted","Data":"e547130ed3c6a3c2b4a430e5bfc1db84ec78249f1aff89d4b3fd822184cd1487"} Jun 02 01:23:58.571633 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:23:58.571612 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f7cf5" event={"ID":"16ba9005-eca6-42a7-96ab-800c12788d7b","Type":"ContainerStarted","Data":"8fe97a4ad9f57ba6a565b449b96bef4a18824479c504c94547ac22c5e271bb89"} Jun 02 01:24:03.582288 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:03.582251 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4vq4w" event={"ID":"ab2a9f80-37fa-4aa6-bc80-1914f1967e78","Type":"ContainerStarted","Data":"3e9a91085d2639c2ae45e0932866e1b33083e9284e2e1ea679e15c45d6a468d8"} Jun 02 01:24:03.583728 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:03.583697 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f7cf5" event={"ID":"16ba9005-eca6-42a7-96ab-800c12788d7b","Type":"ContainerStarted","Data":"34de97a9e316e839646de28c53f47b4cc7c8fb2c27fbe88d18e713779966fef7"} Jun 02 01:24:03.583844 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:03.583824 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:24:03.598813 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:03.598767 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4vq4w" podStartSLOduration=66.315065317 podStartE2EDuration="1m10.598753882s" podCreationTimestamp="2026-06-02 01:22:53 +0000 UTC" firstStartedPulling="2026-06-02 01:23:58.426517838 +0000 UTC m=+66.630978097" lastFinishedPulling="2026-06-02 01:24:02.710206402 +0000 UTC m=+70.914666662" observedRunningTime="2026-06-02 01:24:03.59761422 +0000 UTC m=+71.802074502" watchObservedRunningTime="2026-06-02 01:24:03.598753882 +0000 UTC m=+71.803214164" Jun 02 01:24:03.611385 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:03.611349 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f7cf5" podStartSLOduration=67.235842558 podStartE2EDuration="1m11.611336688s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:23:58.330363313 +0000 UTC m=+66.534823573" lastFinishedPulling="2026-06-02 01:24:02.70585743 +0000 UTC m=+70.910317703" observedRunningTime="2026-06-02 01:24:03.610758197 +0000 UTC m=+71.815218480" watchObservedRunningTime="2026-06-02 01:24:03.611336688 +0000 UTC m=+71.815796970" Jun 02 01:24:30.099603 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:30.099568 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:24:30.100000 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:30.099644 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:24:30.100000 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:30.099767 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 01:24:30.100000 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:30.099849 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert podName:99aff132-a1e2-4586-80cb-cee5ad8238bc nodeName:}" failed. No retries permitted until 2026-06-02 01:25:34.099831917 +0000 UTC m=+162.304292180 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert") pod "ingress-canary-gtr8q" (UID: "99aff132-a1e2-4586-80cb-cee5ad8238bc") : secret "canary-serving-cert" not found Jun 02 01:24:30.100000 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:30.099770 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 01:24:30.100000 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:30.099916 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls podName:0a88a237-04e2-41a9-a1c5-dedfb5bc6df0 nodeName:}" failed. No retries permitted until 2026-06-02 01:25:34.099904901 +0000 UTC m=+162.304365161 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls") pod "dns-default-tfpsh" (UID: "0a88a237-04e2-41a9-a1c5-dedfb5bc6df0") : secret "dns-default-metrics-tls" not found Jun 02 01:24:34.588377 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:34.588347 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f7cf5" Jun 02 01:24:42.421499 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.421467 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z"] Jun 02 01:24:42.423197 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.423180 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.426510 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.426486 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 01:24:42.427313 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.427296 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 02 01:24:42.427372 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.427353 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wvld4\"" Jun 02 01:24:42.427589 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.427572 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 02 01:24:42.427628 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.427587 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 01:24:42.432115 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.432097 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z"] Jun 02 01:24:42.528048 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.528018 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk"] Jun 02 01:24:42.529539 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.529523 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9d94ccc68-rtxfx"] Jun 02 01:24:42.529705 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.529693 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:42.530974 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.530953 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.532004 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.531987 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 02 01:24:42.532004 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.532001 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 02 01:24:42.532212 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.532198 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 02 01:24:42.532280 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.532263 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-689wx\"" Jun 02 01:24:42.535248 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.535232 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 01:24:42.535347 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.535261 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 01:24:42.535513 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.535489 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 01:24:42.535724 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.535710 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-95blv\"" Jun 02 01:24:42.541564 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.541544 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 01:24:42.544121 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.544097 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk"] Jun 02 01:24:42.546012 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.545992 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9d94ccc68-rtxfx"] Jun 02 01:24:42.579584 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.579554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.579584 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.579586 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3e5e657c-d1d2-4183-aae9-86ede8ed3897-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.579776 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.579623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9srnk\" (UniqueName: \"kubernetes.io/projected/3e5e657c-d1d2-4183-aae9-86ede8ed3897-kube-api-access-9srnk\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.628710 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.628680 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5c7cfdd785-nlm5p"] Jun 02 01:24:42.630238 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.630223 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.632266 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.632250 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c7zmc\"" Jun 02 01:24:42.632471 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.632453 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 02 01:24:42.632558 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.632541 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 02 01:24:42.632613 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.632572 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 01:24:42.632613 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.632546 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 02 01:24:42.632869 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.632855 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 02 01:24:42.633213 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.633200 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 01:24:42.646637 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.646615 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c7cfdd785-nlm5p"] Jun 02 01:24:42.680164 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed261979-0c4f-4322-a186-adffe60bffcb-ca-trust-extracted\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.680164 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680137 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9srnk\" (UniqueName: \"kubernetes.io/projected/3e5e657c-d1d2-4183-aae9-86ede8ed3897-kube-api-access-9srnk\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.680308 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-trusted-ca\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.680308 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-bound-sa-token\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.680308 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680263 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.680308 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680282 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-installation-pull-secrets\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.680308 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:42.680528 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680318 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3e5e657c-d1d2-4183-aae9-86ede8ed3897-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.680528 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680337 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-registry-certificates\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.680528 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680353 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm5kk\" (UniqueName: \"kubernetes.io/projected/99fe1df7-c343-44ae-8f29-6a2bd5050426-kube-api-access-tm5kk\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:42.680528 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.680382 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:42.680528 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680410 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-image-registry-private-configuration\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.680528 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680442 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.680528 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.680448 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls podName:3e5e657c-d1d2-4183-aae9-86ede8ed3897 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:43.180433747 +0000 UTC m=+111.384894007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-q5v5z" (UID: "3e5e657c-d1d2-4183-aae9-86ede8ed3897") : secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:42.680528 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.680477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bnb4\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-kube-api-access-6bnb4\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.681115 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.681094 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3e5e657c-d1d2-4183-aae9-86ede8ed3897-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.690410 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.690392 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9srnk\" (UniqueName: \"kubernetes.io/projected/3e5e657c-d1d2-4183-aae9-86ede8ed3897-kube-api-access-9srnk\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:42.781059 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.781225 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781125 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-trusted-ca\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.781225 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781197 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-bound-sa-token\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.781341 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781240 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.781341 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781307 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-844px\" (UniqueName: \"kubernetes.io/projected/16360c44-5de0-4680-99d0-0b9e9120ec63-kube-api-access-844px\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.781444 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781351 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-installation-pull-secrets\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.781556 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781533 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:42.781608 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781576 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-registry-certificates\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.781608 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781601 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tm5kk\" (UniqueName: \"kubernetes.io/projected/99fe1df7-c343-44ae-8f29-6a2bd5050426-kube-api-access-tm5kk\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:42.781747 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781649 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-image-registry-private-configuration\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.781747 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.781703 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 01:24:42.781846 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.781773 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls podName:99fe1df7-c343-44ae-8f29-6a2bd5050426 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:43.281754249 +0000 UTC m=+111.486214508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-bjgpk" (UID: "99fe1df7-c343-44ae-8f29-6a2bd5050426") : secret "samples-operator-tls" not found Jun 02 01:24:42.781846 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781793 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.781846 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781822 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6bnb4\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-kube-api-access-6bnb4\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.782003 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781857 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed261979-0c4f-4322-a186-adffe60bffcb-ca-trust-extracted\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.782003 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-default-certificate\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.782003 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.781934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-stats-auth\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.782003 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.781938 2564 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 01:24:42.782003 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.781955 2564 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d94ccc68-rtxfx: secret "image-registry-tls" not found Jun 02 01:24:42.782233 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.782050 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls podName:ed261979-0c4f-4322-a186-adffe60bffcb nodeName:}" failed. No retries permitted until 2026-06-02 01:24:43.282030815 +0000 UTC m=+111.486491075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls") pod "image-registry-9d94ccc68-rtxfx" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb") : secret "image-registry-tls" not found Jun 02 01:24:42.782291 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.782242 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-registry-certificates\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.782345 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.782291 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed261979-0c4f-4322-a186-adffe60bffcb-ca-trust-extracted\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.782849 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.782828 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-trusted-ca\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.783873 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.783853 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-image-registry-private-configuration\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.783946 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.783920 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-installation-pull-secrets\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.792141 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.792116 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bnb4\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-kube-api-access-6bnb4\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.792296 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.792279 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm5kk\" (UniqueName: \"kubernetes.io/projected/99fe1df7-c343-44ae-8f29-6a2bd5050426-kube-api-access-tm5kk\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:42.792436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.792415 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-bound-sa-token\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:42.883087 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.883050 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-default-certificate\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.883087 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.883091 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-stats-auth\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.883267 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.883109 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.883267 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.883181 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.883267 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.883206 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-844px\" (UniqueName: \"kubernetes.io/projected/16360c44-5de0-4680-99d0-0b9e9120ec63-kube-api-access-844px\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.883415 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.883328 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 01:24:42.883415 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.883358 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:43.383342699 +0000 UTC m=+111.587802975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : configmap references non-existent config key: service-ca.crt Jun 02 01:24:42.883415 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:42.883388 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:43.383370297 +0000 UTC m=+111.587830556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : secret "router-metrics-certs-default" not found Jun 02 01:24:42.885280 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.885260 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-stats-auth\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.885361 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.885301 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-default-certificate\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:42.891586 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:42.891563 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-844px\" (UniqueName: \"kubernetes.io/projected/16360c44-5de0-4680-99d0-0b9e9120ec63-kube-api-access-844px\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:43.184824 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:43.184789 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:43.184996 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.184926 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:43.184996 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.184995 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls podName:3e5e657c-d1d2-4183-aae9-86ede8ed3897 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:44.184978865 +0000 UTC m=+112.389439141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-q5v5z" (UID: "3e5e657c-d1d2-4183-aae9-86ede8ed3897") : secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:43.285704 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:43.285647 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:43.285887 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:43.285738 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:43.285887 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.285793 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 01:24:43.285887 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.285853 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls podName:99fe1df7-c343-44ae-8f29-6a2bd5050426 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:44.285838835 +0000 UTC m=+112.490299095 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-bjgpk" (UID: "99fe1df7-c343-44ae-8f29-6a2bd5050426") : secret "samples-operator-tls" not found Jun 02 01:24:43.285887 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.285869 2564 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 01:24:43.285887 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.285884 2564 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d94ccc68-rtxfx: secret "image-registry-tls" not found Jun 02 01:24:43.286064 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.285957 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls podName:ed261979-0c4f-4322-a186-adffe60bffcb nodeName:}" failed. No retries permitted until 2026-06-02 01:24:44.285942104 +0000 UTC m=+112.490402364 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls") pod "image-registry-9d94ccc68-rtxfx" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb") : secret "image-registry-tls" not found Jun 02 01:24:43.386879 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:43.386848 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:43.386972 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:43.386904 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:43.387012 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.386984 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 01:24:43.387139 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.387019 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:44.387006506 +0000 UTC m=+112.591466766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : configmap references non-existent config key: service-ca.crt Jun 02 01:24:43.387139 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:43.387035 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:44.38702882 +0000 UTC m=+112.591489080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : secret "router-metrics-certs-default" not found Jun 02 01:24:44.193731 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:44.193687 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:44.194086 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.193820 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:44.194086 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.193887 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls podName:3e5e657c-d1d2-4183-aae9-86ede8ed3897 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:46.193871657 +0000 UTC m=+114.398331917 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-q5v5z" (UID: "3e5e657c-d1d2-4183-aae9-86ede8ed3897") : secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:44.294638 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:44.294603 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:44.294751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:44.294726 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:44.294824 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.294785 2564 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 01:24:44.294824 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.294807 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 01:24:44.294824 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.294813 2564 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d94ccc68-rtxfx: secret "image-registry-tls" not found Jun 02 01:24:44.294941 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.294857 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls podName:99fe1df7-c343-44ae-8f29-6a2bd5050426 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:46.294844742 +0000 UTC m=+114.499305002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-bjgpk" (UID: "99fe1df7-c343-44ae-8f29-6a2bd5050426") : secret "samples-operator-tls" not found Jun 02 01:24:44.294941 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.294870 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls podName:ed261979-0c4f-4322-a186-adffe60bffcb nodeName:}" failed. No retries permitted until 2026-06-02 01:24:46.294864042 +0000 UTC m=+114.499324301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls") pod "image-registry-9d94ccc68-rtxfx" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb") : secret "image-registry-tls" not found Jun 02 01:24:44.395959 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:44.395928 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:44.396126 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:44.395986 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:44.396126 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.396078 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 01:24:44.396212 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.396139 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:46.396122014 +0000 UTC m=+114.600582274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : secret "router-metrics-certs-default" not found Jun 02 01:24:44.396212 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:44.396185 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:46.39616049 +0000 UTC m=+114.600620762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : configmap references non-existent config key: service-ca.crt Jun 02 01:24:46.210560 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:46.210520 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:46.210960 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.210680 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:46.210960 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.210744 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls podName:3e5e657c-d1d2-4183-aae9-86ede8ed3897 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:50.210729449 +0000 UTC m=+118.415189714 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-q5v5z" (UID: "3e5e657c-d1d2-4183-aae9-86ede8ed3897") : secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:46.311531 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:46.311500 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:46.311599 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:46.311552 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:46.311695 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.311643 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 01:24:46.311750 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.311741 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls podName:99fe1df7-c343-44ae-8f29-6a2bd5050426 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:50.311726448 +0000 UTC m=+118.516186713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-bjgpk" (UID: "99fe1df7-c343-44ae-8f29-6a2bd5050426") : secret "samples-operator-tls" not found Jun 02 01:24:46.311802 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.311683 2564 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 01:24:46.311802 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.311778 2564 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d94ccc68-rtxfx: secret "image-registry-tls" not found Jun 02 01:24:46.311868 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.311820 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls podName:ed261979-0c4f-4322-a186-adffe60bffcb nodeName:}" failed. No retries permitted until 2026-06-02 01:24:50.31180482 +0000 UTC m=+118.516265082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls") pod "image-registry-9d94ccc68-rtxfx" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb") : secret "image-registry-tls" not found Jun 02 01:24:46.412885 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:46.412849 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:46.413047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:46.412968 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:46.413047 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.413013 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:50.412994504 +0000 UTC m=+118.617454764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : configmap references non-existent config key: service-ca.crt Jun 02 01:24:46.413150 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.413103 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 01:24:46.413150 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:46.413150 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:50.413139644 +0000 UTC m=+118.617599904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : secret "router-metrics-certs-default" not found Jun 02 01:24:50.158049 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:50.158016 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wn6w8_234e4e76-252c-491b-b14f-6b9a7e2c36b9/dns-node-resolver/0.log" Jun 02 01:24:50.245172 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:50.245129 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:50.245348 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.245263 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:50.245348 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.245324 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls podName:3e5e657c-d1d2-4183-aae9-86ede8ed3897 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:58.245310009 +0000 UTC m=+126.449770268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-q5v5z" (UID: "3e5e657c-d1d2-4183-aae9-86ede8ed3897") : secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:50.346105 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:50.346069 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:50.346216 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:50.346125 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:50.346270 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.346237 2564 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 01:24:50.346270 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.346242 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 01:24:50.346416 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.346331 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls podName:99fe1df7-c343-44ae-8f29-6a2bd5050426 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:58.346310096 +0000 UTC m=+126.550770369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-bjgpk" (UID: "99fe1df7-c343-44ae-8f29-6a2bd5050426") : secret "samples-operator-tls" not found Jun 02 01:24:50.346416 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.346248 2564 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d94ccc68-rtxfx: secret "image-registry-tls" not found Jun 02 01:24:50.346416 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.346386 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls podName:ed261979-0c4f-4322-a186-adffe60bffcb nodeName:}" failed. No retries permitted until 2026-06-02 01:24:58.346373127 +0000 UTC m=+126.550833388 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls") pod "image-registry-9d94ccc68-rtxfx" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb") : secret "image-registry-tls" not found Jun 02 01:24:50.447149 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:50.447029 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:50.447149 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:50.447119 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:50.447392 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.447242 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 01:24:50.447392 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.447312 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:58.447293469 +0000 UTC m=+126.651753729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : configmap references non-existent config key: service-ca.crt Jun 02 01:24:50.447392 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:50.447332 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:24:58.447324543 +0000 UTC m=+126.651784807 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : secret "router-metrics-certs-default" not found Jun 02 01:24:50.957471 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:50.957446 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8lt9m_a415cede-b0c5-4300-8f99-22397439fe3f/node-ca/0.log" Jun 02 01:24:58.310328 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.310286 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:24:58.310777 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:58.310432 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:58.310777 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:58.310507 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls podName:3e5e657c-d1d2-4183-aae9-86ede8ed3897 nodeName:}" failed. No retries permitted until 2026-06-02 01:25:14.310492066 +0000 UTC m=+142.514952326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-q5v5z" (UID: "3e5e657c-d1d2-4183-aae9-86ede8ed3897") : secret "cluster-monitoring-operator-tls" not found Jun 02 01:24:58.411299 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.411261 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:58.411431 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.411367 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:58.413603 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.413572 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"image-registry-9d94ccc68-rtxfx\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:58.413732 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.413573 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fe1df7-c343-44ae-8f29-6a2bd5050426-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-bjgpk\" (UID: \"99fe1df7-c343-44ae-8f29-6a2bd5050426\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:58.441785 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.441764 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-689wx\"" Jun 02 01:24:58.446763 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.446747 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-95blv\"" Jun 02 01:24:58.450475 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.450462 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" Jun 02 01:24:58.455101 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.455081 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:58.512144 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.512108 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:58.512259 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.512228 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:58.512979 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:24:58.512558 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle podName:16360c44-5de0-4680-99d0-0b9e9120ec63 nodeName:}" failed. No retries permitted until 2026-06-02 01:25:14.512534415 +0000 UTC m=+142.716994689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle") pod "router-default-5c7cfdd785-nlm5p" (UID: "16360c44-5de0-4680-99d0-0b9e9120ec63") : configmap references non-existent config key: service-ca.crt Jun 02 01:24:58.515626 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.515573 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16360c44-5de0-4680-99d0-0b9e9120ec63-metrics-certs\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:24:58.575835 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.575763 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk"] Jun 02 01:24:58.594051 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.594023 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9d94ccc68-rtxfx"] Jun 02 01:24:58.596739 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:24:58.596714 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded261979_0c4f_4322_a186_adffe60bffcb.slice/crio-a9133499a96545921da2826280426c904ccc60f5979fd719bd9a68ab59707c66 WatchSource:0}: Error finding container a9133499a96545921da2826280426c904ccc60f5979fd719bd9a68ab59707c66: Status 404 returned error can't find the container with id a9133499a96545921da2826280426c904ccc60f5979fd719bd9a68ab59707c66 Jun 02 01:24:58.686138 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.686108 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" event={"ID":"99fe1df7-c343-44ae-8f29-6a2bd5050426","Type":"ContainerStarted","Data":"ae533b2aa1489ea208e887247c399df4d2578353130345e438aabc91126833be"} Jun 02 01:24:58.687161 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.687141 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" event={"ID":"ed261979-0c4f-4322-a186-adffe60bffcb","Type":"ContainerStarted","Data":"445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8"} Jun 02 01:24:58.687253 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.687167 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" event={"ID":"ed261979-0c4f-4322-a186-adffe60bffcb","Type":"ContainerStarted","Data":"a9133499a96545921da2826280426c904ccc60f5979fd719bd9a68ab59707c66"} Jun 02 01:24:58.687295 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.687284 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:24:58.704814 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:24:58.704689 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" podStartSLOduration=16.704674836 podStartE2EDuration="16.704674836s" podCreationTimestamp="2026-06-02 01:24:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 01:24:58.704029765 +0000 UTC m=+126.908490047" watchObservedRunningTime="2026-06-02 01:24:58.704674836 +0000 UTC m=+126.909135119" Jun 02 01:25:00.693060 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:00.692966 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" event={"ID":"99fe1df7-c343-44ae-8f29-6a2bd5050426","Type":"ContainerStarted","Data":"5e8c056179652bb8156f8905b22f4fcf908e544224494f3de18d12d66aae72e6"} Jun 02 01:25:00.693060 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:00.693013 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" event={"ID":"99fe1df7-c343-44ae-8f29-6a2bd5050426","Type":"ContainerStarted","Data":"c242e230ae235fd10408efc99d62d7dcba033f842596b2e80ae3c04f5184c012"} Jun 02 01:25:00.709198 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:00.709151 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-bjgpk" podStartSLOduration=16.933776611 podStartE2EDuration="18.709139463s" podCreationTimestamp="2026-06-02 01:24:42 +0000 UTC" firstStartedPulling="2026-06-02 01:24:58.613308645 +0000 UTC m=+126.817768911" lastFinishedPulling="2026-06-02 01:25:00.3886715 +0000 UTC m=+128.593131763" observedRunningTime="2026-06-02 01:25:00.709028418 +0000 UTC m=+128.913488701" watchObservedRunningTime="2026-06-02 01:25:00.709139463 +0000 UTC m=+128.913599745" Jun 02 01:25:02.140232 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:02.140190 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:25:02.142397 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:02.142376 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf148b2d-3a22-4a94-98f3-5dcc28eebd87-metrics-certs\") pod \"network-metrics-daemon-zvkvb\" (UID: \"cf148b2d-3a22-4a94-98f3-5dcc28eebd87\") " pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:25:02.385952 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:02.385927 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jprs5\"" Jun 02 01:25:02.394700 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:02.394649 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvkvb" Jun 02 01:25:02.507930 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:02.507901 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zvkvb"] Jun 02 01:25:02.510511 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:02.510488 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf148b2d_3a22_4a94_98f3_5dcc28eebd87.slice/crio-db7e7dba27db43230d0bb280f7b516968b7f93c75d2123c5787fd980fb4f8569 WatchSource:0}: Error finding container db7e7dba27db43230d0bb280f7b516968b7f93c75d2123c5787fd980fb4f8569: Status 404 returned error can't find the container with id db7e7dba27db43230d0bb280f7b516968b7f93c75d2123c5787fd980fb4f8569 Jun 02 01:25:02.697472 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:02.697392 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvkvb" event={"ID":"cf148b2d-3a22-4a94-98f3-5dcc28eebd87","Type":"ContainerStarted","Data":"db7e7dba27db43230d0bb280f7b516968b7f93c75d2123c5787fd980fb4f8569"} Jun 02 01:25:04.705914 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:04.705868 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvkvb" event={"ID":"cf148b2d-3a22-4a94-98f3-5dcc28eebd87","Type":"ContainerStarted","Data":"7c2767bd6f826890e400f031cac1f3b04fc1f691f3adb85ac0c2ed51152a996a"} Jun 02 01:25:04.705914 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:04.705911 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvkvb" event={"ID":"cf148b2d-3a22-4a94-98f3-5dcc28eebd87","Type":"ContainerStarted","Data":"962cfaae47c1e7ad4f39cf97270a7e8e5bef12270a77fefee4a4f832a41d41d0"} Jun 02 01:25:09.952906 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.952633 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zvkvb" podStartSLOduration=136.188898026 podStartE2EDuration="2m17.952613068s" podCreationTimestamp="2026-06-02 01:22:52 +0000 UTC" firstStartedPulling="2026-06-02 01:25:02.512262316 +0000 UTC m=+130.716722576" lastFinishedPulling="2026-06-02 01:25:04.275977356 +0000 UTC m=+132.480437618" observedRunningTime="2026-06-02 01:25:04.721142158 +0000 UTC m=+132.925602443" watchObservedRunningTime="2026-06-02 01:25:09.952613068 +0000 UTC m=+138.157073353" Jun 02 01:25:09.953375 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.953117 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zf9vl"] Jun 02 01:25:09.955723 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.955703 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:09.958920 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.958879 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 01:25:09.959055 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.958941 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cxmpn\"" Jun 02 01:25:09.959055 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.958955 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 01:25:09.959055 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.958962 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 01:25:09.959055 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.959021 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 01:25:09.968216 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.968194 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zf9vl"] Jun 02 01:25:09.998912 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.998882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8shh\" (UniqueName: \"kubernetes.io/projected/85025c29-0ce5-42b2-a10f-04fac970bb3f-kube-api-access-l8shh\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:09.999075 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.998916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85025c29-0ce5-42b2-a10f-04fac970bb3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:09.999075 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.998937 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85025c29-0ce5-42b2-a10f-04fac970bb3f-crio-socket\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:09.999182 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.999081 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85025c29-0ce5-42b2-a10f-04fac970bb3f-data-volume\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:09.999182 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:09.999130 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85025c29-0ce5-42b2-a10f-04fac970bb3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.011664 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.011621 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9d94ccc68-rtxfx"] Jun 02 01:25:10.035085 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.035058 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8"] Jun 02 01:25:10.036986 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.036970 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" Jun 02 01:25:10.040182 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.040161 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f8755f774-lbkfj"] Jun 02 01:25:10.041005 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.040983 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xhjrg\"" Jun 02 01:25:10.041444 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.041421 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 02 01:25:10.042198 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.041965 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 02 01:25:10.042707 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.042396 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.045459 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.044773 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-zsj96"] Jun 02 01:25:10.047822 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.047801 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-zsj96" Jun 02 01:25:10.053310 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.053294 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 01:25:10.055806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.055788 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 01:25:10.060040 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.060024 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-b9lnn\"" Jun 02 01:25:10.067744 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.067724 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8"] Jun 02 01:25:10.084779 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.084729 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-zsj96"] Jun 02 01:25:10.085599 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.085576 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f8755f774-lbkfj"] Jun 02 01:25:10.099836 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.099809 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85025c29-0ce5-42b2-a10f-04fac970bb3f-crio-socket\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.099940 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.099848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-bound-sa-token\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.099940 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.099878 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0362e1c2-f61f-4871-a49a-348ad82648e6-image-registry-private-configuration\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.099940 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.099921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0362e1c2-f61f-4871-a49a-348ad82648e6-trusted-ca\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.099940 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.099934 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85025c29-0ce5-42b2-a10f-04fac970bb3f-crio-socket\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.100143 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.099963 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fb11ade2-f6f5-4fea-bf71-8193a5071c03-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-qm5j8\" (UID: \"fb11ade2-f6f5-4fea-bf71-8193a5071c03\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" Jun 02 01:25:10.100143 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.099991 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7gmb\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-kube-api-access-f7gmb\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.100143 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100046 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85025c29-0ce5-42b2-a10f-04fac970bb3f-data-volume\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.100143 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100083 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0362e1c2-f61f-4871-a49a-348ad82648e6-ca-trust-extracted\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.100143 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100111 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0362e1c2-f61f-4871-a49a-348ad82648e6-registry-certificates\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.100337 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100140 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85025c29-0ce5-42b2-a10f-04fac970bb3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.100337 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fb11ade2-f6f5-4fea-bf71-8193a5071c03-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-qm5j8\" (UID: \"fb11ade2-f6f5-4fea-bf71-8193a5071c03\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" Jun 02 01:25:10.100337 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100229 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0362e1c2-f61f-4871-a49a-348ad82648e6-installation-pull-secrets\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.100337 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100264 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l8shh\" (UniqueName: \"kubernetes.io/projected/85025c29-0ce5-42b2-a10f-04fac970bb3f-kube-api-access-l8shh\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.100337 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-registry-tls\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.100337 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100296 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85025c29-0ce5-42b2-a10f-04fac970bb3f-data-volume\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.100337 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100304 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85025c29-0ce5-42b2-a10f-04fac970bb3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.100337 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100328 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t485q\" (UniqueName: \"kubernetes.io/projected/097f5758-d818-44c7-bd95-39a8e63b75e0-kube-api-access-t485q\") pod \"downloads-c65b8dc4b-zsj96\" (UID: \"097f5758-d818-44c7-bd95-39a8e63b75e0\") " pod="openshift-console/downloads-c65b8dc4b-zsj96" Jun 02 01:25:10.100748 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.100734 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85025c29-0ce5-42b2-a10f-04fac970bb3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.102336 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.102318 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85025c29-0ce5-42b2-a10f-04fac970bb3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.110802 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.110778 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8shh\" (UniqueName: \"kubernetes.io/projected/85025c29-0ce5-42b2-a10f-04fac970bb3f-kube-api-access-l8shh\") pod \"insights-runtime-extractor-zf9vl\" (UID: \"85025c29-0ce5-42b2-a10f-04fac970bb3f\") " pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.200849 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.200818 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0362e1c2-f61f-4871-a49a-348ad82648e6-ca-trust-extracted\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.200849 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.200851 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0362e1c2-f61f-4871-a49a-348ad82648e6-registry-certificates\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.201072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.200872 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fb11ade2-f6f5-4fea-bf71-8193a5071c03-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-qm5j8\" (UID: \"fb11ade2-f6f5-4fea-bf71-8193a5071c03\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" Jun 02 01:25:10.201072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.200918 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0362e1c2-f61f-4871-a49a-348ad82648e6-installation-pull-secrets\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.201072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.200960 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-registry-tls\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.201226 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.201115 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t485q\" (UniqueName: \"kubernetes.io/projected/097f5758-d818-44c7-bd95-39a8e63b75e0-kube-api-access-t485q\") pod \"downloads-c65b8dc4b-zsj96\" (UID: \"097f5758-d818-44c7-bd95-39a8e63b75e0\") " pod="openshift-console/downloads-c65b8dc4b-zsj96" Jun 02 01:25:10.201226 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.201167 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-bound-sa-token\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.201226 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.201194 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0362e1c2-f61f-4871-a49a-348ad82648e6-image-registry-private-configuration\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.201362 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.201231 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0362e1c2-f61f-4871-a49a-348ad82648e6-trusted-ca\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.201362 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.201274 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fb11ade2-f6f5-4fea-bf71-8193a5071c03-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-qm5j8\" (UID: \"fb11ade2-f6f5-4fea-bf71-8193a5071c03\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" Jun 02 01:25:10.201362 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.201287 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0362e1c2-f61f-4871-a49a-348ad82648e6-ca-trust-extracted\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.201362 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.201306 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f7gmb\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-kube-api-access-f7gmb\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.201810 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.201781 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0362e1c2-f61f-4871-a49a-348ad82648e6-registry-certificates\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.202029 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.202008 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fb11ade2-f6f5-4fea-bf71-8193a5071c03-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-qm5j8\" (UID: \"fb11ade2-f6f5-4fea-bf71-8193a5071c03\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" Jun 02 01:25:10.202362 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.202340 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0362e1c2-f61f-4871-a49a-348ad82648e6-trusted-ca\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.203481 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.203434 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0362e1c2-f61f-4871-a49a-348ad82648e6-installation-pull-secrets\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.203569 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.203551 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-registry-tls\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.203975 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.203956 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0362e1c2-f61f-4871-a49a-348ad82648e6-image-registry-private-configuration\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.204047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.203973 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fb11ade2-f6f5-4fea-bf71-8193a5071c03-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-qm5j8\" (UID: \"fb11ade2-f6f5-4fea-bf71-8193a5071c03\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" Jun 02 01:25:10.214432 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.214410 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t485q\" (UniqueName: \"kubernetes.io/projected/097f5758-d818-44c7-bd95-39a8e63b75e0-kube-api-access-t485q\") pod \"downloads-c65b8dc4b-zsj96\" (UID: \"097f5758-d818-44c7-bd95-39a8e63b75e0\") " pod="openshift-console/downloads-c65b8dc4b-zsj96" Jun 02 01:25:10.214599 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.214580 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7gmb\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-kube-api-access-f7gmb\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.214782 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.214768 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0362e1c2-f61f-4871-a49a-348ad82648e6-bound-sa-token\") pod \"image-registry-7f8755f774-lbkfj\" (UID: \"0362e1c2-f61f-4871-a49a-348ad82648e6\") " pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.264634 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.264602 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zf9vl" Jun 02 01:25:10.350175 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.350151 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" Jun 02 01:25:10.356936 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.356910 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.360756 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.360647 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-zsj96" Jun 02 01:25:10.384769 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.384713 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zf9vl"] Jun 02 01:25:10.388975 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:10.388948 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85025c29_0ce5_42b2_a10f_04fac970bb3f.slice/crio-912fb9d40386d2ae8f7b3841d4c1ac8f2c91de7739c528f42af11c900951e1bc WatchSource:0}: Error finding container 912fb9d40386d2ae8f7b3841d4c1ac8f2c91de7739c528f42af11c900951e1bc: Status 404 returned error can't find the container with id 912fb9d40386d2ae8f7b3841d4c1ac8f2c91de7739c528f42af11c900951e1bc Jun 02 01:25:10.502484 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.501592 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f8755f774-lbkfj"] Jun 02 01:25:10.506469 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:10.506441 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0362e1c2_f61f_4871_a49a_348ad82648e6.slice/crio-2b9870f539ce2cb37c7e430a4e9338181f2b1e06a4c0b49c9282d1858edbae29 WatchSource:0}: Error finding container 2b9870f539ce2cb37c7e430a4e9338181f2b1e06a4c0b49c9282d1858edbae29: Status 404 returned error can't find the container with id 2b9870f539ce2cb37c7e430a4e9338181f2b1e06a4c0b49c9282d1858edbae29 Jun 02 01:25:10.512496 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.512424 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8"] Jun 02 01:25:10.518008 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:10.517983 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb11ade2_f6f5_4fea_bf71_8193a5071c03.slice/crio-9bf6d08b1d812924dd3ec5f2ffb6a01e35703a2f8409c4b87b4f2f18fb9ff7af WatchSource:0}: Error finding container 9bf6d08b1d812924dd3ec5f2ffb6a01e35703a2f8409c4b87b4f2f18fb9ff7af: Status 404 returned error can't find the container with id 9bf6d08b1d812924dd3ec5f2ffb6a01e35703a2f8409c4b87b4f2f18fb9ff7af Jun 02 01:25:10.537038 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.536875 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-zsj96"] Jun 02 01:25:10.539318 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:10.539294 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097f5758_d818_44c7_bd95_39a8e63b75e0.slice/crio-31c58dfc0fc4eec1c1641df58144acd871daecde92aa59bac8047f6122bfcafe WatchSource:0}: Error finding container 31c58dfc0fc4eec1c1641df58144acd871daecde92aa59bac8047f6122bfcafe: Status 404 returned error can't find the container with id 31c58dfc0fc4eec1c1641df58144acd871daecde92aa59bac8047f6122bfcafe Jun 02 01:25:10.722918 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.722832 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-zsj96" event={"ID":"097f5758-d818-44c7-bd95-39a8e63b75e0","Type":"ContainerStarted","Data":"31c58dfc0fc4eec1c1641df58144acd871daecde92aa59bac8047f6122bfcafe"} Jun 02 01:25:10.723751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.723731 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" event={"ID":"fb11ade2-f6f5-4fea-bf71-8193a5071c03","Type":"ContainerStarted","Data":"9bf6d08b1d812924dd3ec5f2ffb6a01e35703a2f8409c4b87b4f2f18fb9ff7af"} Jun 02 01:25:10.724997 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.724972 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zf9vl" event={"ID":"85025c29-0ce5-42b2-a10f-04fac970bb3f","Type":"ContainerStarted","Data":"c2bb58da35d380f8872c8eb9ad8c121029b5c46d9fa1251d36304dc3eebbd0cc"} Jun 02 01:25:10.725074 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.725004 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zf9vl" event={"ID":"85025c29-0ce5-42b2-a10f-04fac970bb3f","Type":"ContainerStarted","Data":"912fb9d40386d2ae8f7b3841d4c1ac8f2c91de7739c528f42af11c900951e1bc"} Jun 02 01:25:10.726243 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.726226 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" event={"ID":"0362e1c2-f61f-4871-a49a-348ad82648e6","Type":"ContainerStarted","Data":"f3db03454de84581ec6b1a2856839ce39c8902cba1caf208a69df440641d4cf1"} Jun 02 01:25:10.726316 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.726247 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" event={"ID":"0362e1c2-f61f-4871-a49a-348ad82648e6","Type":"ContainerStarted","Data":"2b9870f539ce2cb37c7e430a4e9338181f2b1e06a4c0b49c9282d1858edbae29"} Jun 02 01:25:10.726379 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.726363 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:10.745923 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:10.745872 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" podStartSLOduration=0.745859927 podStartE2EDuration="745.859927ms" podCreationTimestamp="2026-06-02 01:25:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 01:25:10.744744269 +0000 UTC m=+138.949204550" watchObservedRunningTime="2026-06-02 01:25:10.745859927 +0000 UTC m=+138.950320202" Jun 02 01:25:11.731202 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:11.731081 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zf9vl" event={"ID":"85025c29-0ce5-42b2-a10f-04fac970bb3f","Type":"ContainerStarted","Data":"b7e6b8df66ea63acaaf40c14a70a4aa1e24fd695024f30d19228dbba4398fdf5"} Jun 02 01:25:12.734833 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:12.734794 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" event={"ID":"fb11ade2-f6f5-4fea-bf71-8193a5071c03","Type":"ContainerStarted","Data":"de235bb8b6a898cac999e567162412089729b66bb912a3eea22ea7d0bae6d4bd"} Jun 02 01:25:12.750161 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:12.749943 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-qm5j8" podStartSLOduration=1.5443978390000002 podStartE2EDuration="2.749926599s" podCreationTimestamp="2026-06-02 01:25:10 +0000 UTC" firstStartedPulling="2026-06-02 01:25:10.519795571 +0000 UTC m=+138.724255831" lastFinishedPulling="2026-06-02 01:25:11.725324331 +0000 UTC m=+139.929784591" observedRunningTime="2026-06-02 01:25:12.749396591 +0000 UTC m=+140.953856873" watchObservedRunningTime="2026-06-02 01:25:12.749926599 +0000 UTC m=+140.954386883" Jun 02 01:25:14.338693 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.338640 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:25:14.341003 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.340976 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e5e657c-d1d2-4183-aae9-86ede8ed3897-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-q5v5z\" (UID: \"3e5e657c-d1d2-4183-aae9-86ede8ed3897\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:25:14.533960 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.533881 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wvld4\"" Jun 02 01:25:14.539853 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.539832 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:25:14.540387 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.540369 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16360c44-5de0-4680-99d0-0b9e9120ec63-service-ca-bundle\") pod \"router-default-5c7cfdd785-nlm5p\" (UID: \"16360c44-5de0-4680-99d0-0b9e9120ec63\") " pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:25:14.542793 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.542777 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" Jun 02 01:25:14.680461 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.680426 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z"] Jun 02 01:25:14.683406 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:14.683378 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e5e657c_d1d2_4183_aae9_86ede8ed3897.slice/crio-b7232fa7a9a35821946f32ec336703d03b0b3ae73ea6d9c40b361834ee23ad13 WatchSource:0}: Error finding container b7232fa7a9a35821946f32ec336703d03b0b3ae73ea6d9c40b361834ee23ad13: Status 404 returned error can't find the container with id b7232fa7a9a35821946f32ec336703d03b0b3ae73ea6d9c40b361834ee23ad13 Jun 02 01:25:14.740711 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.740682 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c7zmc\"" Jun 02 01:25:14.743122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.743052 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zf9vl" event={"ID":"85025c29-0ce5-42b2-a10f-04fac970bb3f","Type":"ContainerStarted","Data":"fd6fb6c1ac4ce26a49344c26dd5fde976a68c6dcea19a65c3c3b302f0f108aec"} Jun 02 01:25:14.744223 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.744186 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" event={"ID":"3e5e657c-d1d2-4183-aae9-86ede8ed3897","Type":"ContainerStarted","Data":"b7232fa7a9a35821946f32ec336703d03b0b3ae73ea6d9c40b361834ee23ad13"} Jun 02 01:25:14.748639 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.748603 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:25:14.758835 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.758785 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zf9vl" podStartSLOduration=2.043201803 podStartE2EDuration="5.758767503s" podCreationTimestamp="2026-06-02 01:25:09 +0000 UTC" firstStartedPulling="2026-06-02 01:25:10.474426139 +0000 UTC m=+138.678886413" lastFinishedPulling="2026-06-02 01:25:14.189991839 +0000 UTC m=+142.394452113" observedRunningTime="2026-06-02 01:25:14.758075158 +0000 UTC m=+142.962535441" watchObservedRunningTime="2026-06-02 01:25:14.758767503 +0000 UTC m=+142.963227784" Jun 02 01:25:14.881857 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:14.881821 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c7cfdd785-nlm5p"] Jun 02 01:25:14.887392 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:14.887366 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16360c44_5de0_4680_99d0_0b9e9120ec63.slice/crio-0d2bb73ea8149c4cd1e47d7ca9208515cee6325fa8b0e797d0fff5e3567fbfc1 WatchSource:0}: Error finding container 0d2bb73ea8149c4cd1e47d7ca9208515cee6325fa8b0e797d0fff5e3567fbfc1: Status 404 returned error can't find the container with id 0d2bb73ea8149c4cd1e47d7ca9208515cee6325fa8b0e797d0fff5e3567fbfc1 Jun 02 01:25:15.752073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:15.752039 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" event={"ID":"16360c44-5de0-4680-99d0-0b9e9120ec63","Type":"ContainerStarted","Data":"ab017733aadbacd05b2849a3c42d90b85bc2a38583ddf90c1c3659c5f80b2330"} Jun 02 01:25:15.752073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:15.752075 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" event={"ID":"16360c44-5de0-4680-99d0-0b9e9120ec63","Type":"ContainerStarted","Data":"0d2bb73ea8149c4cd1e47d7ca9208515cee6325fa8b0e797d0fff5e3567fbfc1"} Jun 02 01:25:15.770063 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:15.770011 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" podStartSLOduration=33.769997796 podStartE2EDuration="33.769997796s" podCreationTimestamp="2026-06-02 01:24:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 01:25:15.769061848 +0000 UTC m=+143.973522130" watchObservedRunningTime="2026-06-02 01:25:15.769997796 +0000 UTC m=+143.974458077" Jun 02 01:25:16.749193 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:16.749157 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:25:16.751631 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:16.751609 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:25:16.754857 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:16.754796 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:25:16.755856 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:16.755832 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5c7cfdd785-nlm5p" Jun 02 01:25:17.760791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:17.760751 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" event={"ID":"3e5e657c-d1d2-4183-aae9-86ede8ed3897","Type":"ContainerStarted","Data":"437b2b6ea346d3bb158902409abd3e861edafe215bb2e58b0285aa2dd3e12002"} Jun 02 01:25:17.778189 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:17.778139 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-q5v5z" podStartSLOduration=33.67059946 podStartE2EDuration="35.778125065s" podCreationTimestamp="2026-06-02 01:24:42 +0000 UTC" firstStartedPulling="2026-06-02 01:25:14.685607873 +0000 UTC m=+142.890068140" lastFinishedPulling="2026-06-02 01:25:16.79313347 +0000 UTC m=+144.997593745" observedRunningTime="2026-06-02 01:25:17.777217071 +0000 UTC m=+145.981677354" watchObservedRunningTime="2026-06-02 01:25:17.778125065 +0000 UTC m=+145.982585352" Jun 02 01:25:20.017050 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:20.016996 2564 patch_prober.go:28] interesting pod/image-registry-9d94ccc68-rtxfx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 01:25:20.017404 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:20.017073 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" podUID="ed261979-0c4f-4322-a186-adffe60bffcb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 01:25:26.739496 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.739458 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt"] Jun 02 01:25:26.745362 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.745336 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.747890 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.747864 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gppph\"" Jun 02 01:25:26.748017 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.747948 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 02 01:25:26.748078 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.748040 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 02 01:25:26.748938 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.748915 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 01:25:26.749165 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.749110 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 02 01:25:26.755222 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.755189 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt"] Jun 02 01:25:26.756694 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.756672 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rrk44"] Jun 02 01:25:26.760882 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.760861 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.763752 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.763724 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 01:25:26.763752 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.763744 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n7ndj\"" Jun 02 01:25:26.764007 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.763991 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 01:25:26.764577 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.764556 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 01:25:26.848564 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-tls\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.848753 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b4e7c0-a11b-4424-9664-a14c00c94b4e-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.848753 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848616 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-textfile\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.848753 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.848753 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848680 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51bbd087-24d1-475d-ba30-bedf031a883c-metrics-client-ca\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.848980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848767 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.848980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848810 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.848980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848840 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-accelerators-collector-config\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.848980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sg2z\" (UniqueName: \"kubernetes.io/projected/51bbd087-24d1-475d-ba30-bedf031a883c-kube-api-access-2sg2z\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.848980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848922 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.848980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-wtmp\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.848980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.848977 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-root\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.849269 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.849017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/61b4e7c0-a11b-4424-9664-a14c00c94b4e-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.849269 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.849043 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-sys\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.849269 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.849076 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvgs7\" (UniqueName: \"kubernetes.io/projected/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-api-access-wvgs7\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.950335 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950299 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-tls\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.950502 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950349 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b4e7c0-a11b-4424-9664-a14c00c94b4e-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.950502 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950394 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-textfile\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.950502 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950424 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.950502 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950454 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51bbd087-24d1-475d-ba30-bedf031a883c-metrics-client-ca\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950526 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950559 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950586 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-accelerators-collector-config\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950612 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2sg2z\" (UniqueName: \"kubernetes.io/projected/51bbd087-24d1-475d-ba30-bedf031a883c-kube-api-access-2sg2z\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950707 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950736 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-wtmp\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950770 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-root\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950808 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/61b4e7c0-a11b-4424-9664-a14c00c94b4e-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.950858 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950834 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-sys\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.951312 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.950870 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wvgs7\" (UniqueName: \"kubernetes.io/projected/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-api-access-wvgs7\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.951312 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951207 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b4e7c0-a11b-4424-9664-a14c00c94b4e-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.951312 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951271 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.951312 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951304 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-textfile\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.951509 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951325 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-accelerators-collector-config\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.951509 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951350 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-root\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.951509 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:25:26.951431 2564 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 02 01:25:26.951509 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951464 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-sys\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.951509 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951488 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-wtmp\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.951769 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:25:26.951510 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-tls podName:61b4e7c0-a11b-4424-9664-a14c00c94b4e nodeName:}" failed. No retries permitted until 2026-06-02 01:25:27.451492411 +0000 UTC m=+155.655952688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-tls") pod "kube-state-metrics-8469c486bb-8gnlt" (UID: "61b4e7c0-a11b-4424-9664-a14c00c94b4e") : secret "kube-state-metrics-tls" not found Jun 02 01:25:26.951769 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951742 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/61b4e7c0-a11b-4424-9664-a14c00c94b4e-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.951885 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.951843 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51bbd087-24d1-475d-ba30-bedf031a883c-metrics-client-ca\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.954089 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.954062 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.954227 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.954101 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/51bbd087-24d1-475d-ba30-bedf031a883c-node-exporter-tls\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:26.954227 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.954132 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.960936 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.960908 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvgs7\" (UniqueName: \"kubernetes.io/projected/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-api-access-wvgs7\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:26.961247 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:26.961223 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sg2z\" (UniqueName: \"kubernetes.io/projected/51bbd087-24d1-475d-ba30-bedf031a883c-kube-api-access-2sg2z\") pod \"node-exporter-rrk44\" (UID: \"51bbd087-24d1-475d-ba30-bedf031a883c\") " pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:27.074349 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:27.074275 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rrk44" Jun 02 01:25:27.453784 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:27.453747 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:27.456470 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:27.456442 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b4e7c0-a11b-4424-9664-a14c00c94b4e-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-8gnlt\" (UID: \"61b4e7c0-a11b-4424-9664-a14c00c94b4e\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:27.656237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:27.656202 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" Jun 02 01:25:29.213040 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:25:29.212991 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tfpsh" podUID="0a88a237-04e2-41a9-a1c5-dedfb5bc6df0" Jun 02 01:25:29.218126 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:25:29.218094 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-gtr8q" podUID="99aff132-a1e2-4586-80cb-cee5ad8238bc" Jun 02 01:25:29.529740 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:29.529572 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51bbd087_24d1_475d_ba30_bedf031a883c.slice/crio-e11e23625ab6d97d99031577f3bdf02dddc0594541e187776cad8174d055af7c WatchSource:0}: Error finding container e11e23625ab6d97d99031577f3bdf02dddc0594541e187776cad8174d055af7c: Status 404 returned error can't find the container with id e11e23625ab6d97d99031577f3bdf02dddc0594541e187776cad8174d055af7c Jun 02 01:25:29.655505 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.655481 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt"] Jun 02 01:25:29.664708 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:29.664680 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b4e7c0_a11b_4424_9664_a14c00c94b4e.slice/crio-a470b70f59dfc0495aef03e68063f62d01837c4be9038dde77a47ce9d28655bc WatchSource:0}: Error finding container a470b70f59dfc0495aef03e68063f62d01837c4be9038dde77a47ce9d28655bc: Status 404 returned error can't find the container with id a470b70f59dfc0495aef03e68063f62d01837c4be9038dde77a47ce9d28655bc Jun 02 01:25:29.714620 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.714591 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78987677b8-nptt4"] Jun 02 01:25:29.719033 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.719014 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.721608 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.721586 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 02 01:25:29.721608 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.721602 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 02 01:25:29.721786 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.721586 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 02 01:25:29.721926 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.721908 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e1tfaavum9cp2\"" Jun 02 01:25:29.722031 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.721948 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7kbbz\"" Jun 02 01:25:29.722098 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.722070 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 02 01:25:29.722187 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.722172 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 02 01:25:29.729739 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.729716 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78987677b8-nptt4"] Jun 02 01:25:29.776494 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.776411 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.776494 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.776452 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91ac6667-5311-4e0a-93a1-dad9f25b6713-metrics-client-ca\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.776494 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.776484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.776791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.776554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-grpc-tls\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.776791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.776635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl24m\" (UniqueName: \"kubernetes.io/projected/91ac6667-5311-4e0a-93a1-dad9f25b6713-kube-api-access-sl24m\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.776791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.776673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.776791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.776708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.776791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.776778 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-tls\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.792526 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.792492 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-zsj96" event={"ID":"097f5758-d818-44c7-bd95-39a8e63b75e0","Type":"ContainerStarted","Data":"d276ca995ba0978afa9d4fe3e6928cf41bc61ef0357217d9b14b90bb6d6cd09a"} Jun 02 01:25:29.792706 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.792689 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-zsj96" Jun 02 01:25:29.793716 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.793686 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rrk44" event={"ID":"51bbd087-24d1-475d-ba30-bedf031a883c","Type":"ContainerStarted","Data":"e11e23625ab6d97d99031577f3bdf02dddc0594541e187776cad8174d055af7c"} Jun 02 01:25:29.794285 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.794266 2564 patch_prober.go:28] interesting pod/downloads-c65b8dc4b-zsj96 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.15:8080/\": dial tcp 10.133.0.15:8080: connect: connection refused" start-of-body= Jun 02 01:25:29.794374 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.794308 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-c65b8dc4b-zsj96" podUID="097f5758-d818-44c7-bd95-39a8e63b75e0" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.15:8080/\": dial tcp 10.133.0.15:8080: connect: connection refused" Jun 02 01:25:29.794894 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.794878 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:25:29.794981 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.794875 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" event={"ID":"61b4e7c0-a11b-4424-9664-a14c00c94b4e","Type":"ContainerStarted","Data":"a470b70f59dfc0495aef03e68063f62d01837c4be9038dde77a47ce9d28655bc"} Jun 02 01:25:29.795039 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.795001 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tfpsh" Jun 02 01:25:29.808405 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.808366 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-zsj96" podStartSLOduration=0.723677506 podStartE2EDuration="19.808352955s" podCreationTimestamp="2026-06-02 01:25:10 +0000 UTC" firstStartedPulling="2026-06-02 01:25:10.541033051 +0000 UTC m=+138.745493311" lastFinishedPulling="2026-06-02 01:25:29.625708483 +0000 UTC m=+157.830168760" observedRunningTime="2026-06-02 01:25:29.807539509 +0000 UTC m=+158.011999793" watchObservedRunningTime="2026-06-02 01:25:29.808352955 +0000 UTC m=+158.012813610" Jun 02 01:25:29.878010 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.877969 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-tls\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.878175 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.878018 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.878175 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.878065 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91ac6667-5311-4e0a-93a1-dad9f25b6713-metrics-client-ca\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.878175 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.878111 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.878175 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.878154 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-grpc-tls\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.878401 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.878245 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sl24m\" (UniqueName: \"kubernetes.io/projected/91ac6667-5311-4e0a-93a1-dad9f25b6713-kube-api-access-sl24m\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.878401 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.878277 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.878401 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.878321 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.878969 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.878947 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91ac6667-5311-4e0a-93a1-dad9f25b6713-metrics-client-ca\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.881377 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.881333 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-tls\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.882147 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.882089 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.882284 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.882249 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.882361 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.882304 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.882361 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.882336 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-grpc-tls\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.882709 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.882683 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91ac6667-5311-4e0a-93a1-dad9f25b6713-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:29.892201 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:29.892120 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl24m\" (UniqueName: \"kubernetes.io/projected/91ac6667-5311-4e0a-93a1-dad9f25b6713-kube-api-access-sl24m\") pod \"thanos-querier-78987677b8-nptt4\" (UID: \"91ac6667-5311-4e0a-93a1-dad9f25b6713\") " pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:30.018449 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:30.018376 2564 patch_prober.go:28] interesting pod/image-registry-9d94ccc68-rtxfx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 01:25:30.018630 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:30.018456 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" podUID="ed261979-0c4f-4322-a186-adffe60bffcb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 01:25:30.029196 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:30.028744 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:30.184987 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:30.184325 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78987677b8-nptt4"] Jun 02 01:25:30.192416 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:30.192226 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ac6667_5311_4e0a_93a1_dad9f25b6713.slice/crio-24c7dcdf0623f07356bf680920f129e25b3d9a593ce660f034042c6f7a5e64f9 WatchSource:0}: Error finding container 24c7dcdf0623f07356bf680920f129e25b3d9a593ce660f034042c6f7a5e64f9: Status 404 returned error can't find the container with id 24c7dcdf0623f07356bf680920f129e25b3d9a593ce660f034042c6f7a5e64f9 Jun 02 01:25:30.372325 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:30.369496 2564 patch_prober.go:28] interesting pod/image-registry-7f8755f774-lbkfj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 01:25:30.372325 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:30.369563 2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" podUID="0362e1c2-f61f-4871-a49a-348ad82648e6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 01:25:30.799475 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:30.799387 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" event={"ID":"91ac6667-5311-4e0a-93a1-dad9f25b6713","Type":"ContainerStarted","Data":"24c7dcdf0623f07356bf680920f129e25b3d9a593ce660f034042c6f7a5e64f9"} Jun 02 01:25:30.804370 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:30.804342 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-zsj96" Jun 02 01:25:31.038265 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.038225 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-675b845476-klc5n"] Jun 02 01:25:31.057138 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.056182 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-675b845476-klc5n"] Jun 02 01:25:31.057138 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.056296 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.059276 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.059252 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 01:25:31.059601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.059444 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 02 01:25:31.059601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.059485 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 02 01:25:31.060001 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.059771 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 02 01:25:31.060001 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.059822 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mpkpg\"" Jun 02 01:25:31.060457 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.060229 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-16iv3k784eai0\"" Jun 02 01:25:31.192620 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.192585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/762b97d5-b17b-432b-be89-715419a3d70f-metrics-server-audit-profiles\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.192888 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.192641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/762b97d5-b17b-432b-be89-715419a3d70f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.192888 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.192704 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-secret-metrics-server-client-certs\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.192888 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.192738 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-client-ca-bundle\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.192888 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.192782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-secret-metrics-server-tls\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.192888 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.192810 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/762b97d5-b17b-432b-be89-715419a3d70f-audit-log\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.192888 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.192841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6nwb\" (UniqueName: \"kubernetes.io/projected/762b97d5-b17b-432b-be89-715419a3d70f-kube-api-access-w6nwb\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.294464 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.293639 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-client-ca-bundle\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.294464 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.293719 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-secret-metrics-server-tls\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.294464 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.293748 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/762b97d5-b17b-432b-be89-715419a3d70f-audit-log\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.294464 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.293786 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w6nwb\" (UniqueName: \"kubernetes.io/projected/762b97d5-b17b-432b-be89-715419a3d70f-kube-api-access-w6nwb\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.294464 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.293900 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/762b97d5-b17b-432b-be89-715419a3d70f-metrics-server-audit-profiles\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.294464 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.293932 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/762b97d5-b17b-432b-be89-715419a3d70f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.294464 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.293973 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-secret-metrics-server-client-certs\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.295129 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.295031 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/762b97d5-b17b-432b-be89-715419a3d70f-audit-log\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.298607 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.297844 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/762b97d5-b17b-432b-be89-715419a3d70f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.299796 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.299559 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-client-ca-bundle\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.299796 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.299744 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/762b97d5-b17b-432b-be89-715419a3d70f-metrics-server-audit-profiles\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.302393 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.302351 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-secret-metrics-server-tls\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.303620 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.303594 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/762b97d5-b17b-432b-be89-715419a3d70f-secret-metrics-server-client-certs\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.307448 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.307377 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6nwb\" (UniqueName: \"kubernetes.io/projected/762b97d5-b17b-432b-be89-715419a3d70f-kube-api-access-w6nwb\") pod \"metrics-server-675b845476-klc5n\" (UID: \"762b97d5-b17b-432b-be89-715419a3d70f\") " pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.370629 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.370126 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:31.519746 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.519707 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt"] Jun 02 01:25:31.541261 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.540172 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt"] Jun 02 01:25:31.541261 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.540330 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" Jun 02 01:25:31.543381 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.543304 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-96mln\"" Jun 02 01:25:31.543610 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.543590 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 02 01:25:31.598375 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.598241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d3ad6a3c-8841-4dd1-9309-91975b7ff7c8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-wnwvt\" (UID: \"d3ad6a3c-8841-4dd1-9309-91975b7ff7c8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" Jun 02 01:25:31.699407 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.699358 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d3ad6a3c-8841-4dd1-9309-91975b7ff7c8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-wnwvt\" (UID: \"d3ad6a3c-8841-4dd1-9309-91975b7ff7c8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" Jun 02 01:25:31.699577 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:25:31.699543 2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 02 01:25:31.699713 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:25:31.699624 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3ad6a3c-8841-4dd1-9309-91975b7ff7c8-monitoring-plugin-cert podName:d3ad6a3c-8841-4dd1-9309-91975b7ff7c8 nodeName:}" failed. No retries permitted until 2026-06-02 01:25:32.199603244 +0000 UTC m=+160.404063520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/d3ad6a3c-8841-4dd1-9309-91975b7ff7c8-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-wnwvt" (UID: "d3ad6a3c-8841-4dd1-9309-91975b7ff7c8") : secret "monitoring-plugin-cert" not found Jun 02 01:25:31.722924 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.722586 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-675b845476-klc5n"] Jun 02 01:25:31.728685 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:31.728634 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762b97d5_b17b_432b_be89_715419a3d70f.slice/crio-92dd25c62c972b68edd2694ee3dbfe66ba509a34d768bbe75254742bdc80c425 WatchSource:0}: Error finding container 92dd25c62c972b68edd2694ee3dbfe66ba509a34d768bbe75254742bdc80c425: Status 404 returned error can't find the container with id 92dd25c62c972b68edd2694ee3dbfe66ba509a34d768bbe75254742bdc80c425 Jun 02 01:25:31.737236 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.737127 2564 patch_prober.go:28] interesting pod/image-registry-7f8755f774-lbkfj container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 01:25:31.737236 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.737175 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" podUID="0362e1c2-f61f-4871-a49a-348ad82648e6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 01:25:31.808796 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.808742 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-675b845476-klc5n" event={"ID":"762b97d5-b17b-432b-be89-715419a3d70f","Type":"ContainerStarted","Data":"92dd25c62c972b68edd2694ee3dbfe66ba509a34d768bbe75254742bdc80c425"} Jun 02 01:25:31.810895 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:31.810820 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rrk44" event={"ID":"51bbd087-24d1-475d-ba30-bedf031a883c","Type":"ContainerStarted","Data":"bd7336f5c4f3d4f0cf07da15862692e75fef72ffb366b0f878b05d72abc91c0c"} Jun 02 01:25:32.205064 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.205021 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d3ad6a3c-8841-4dd1-9309-91975b7ff7c8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-wnwvt\" (UID: \"d3ad6a3c-8841-4dd1-9309-91975b7ff7c8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" Jun 02 01:25:32.208820 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.208786 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d3ad6a3c-8841-4dd1-9309-91975b7ff7c8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-wnwvt\" (UID: \"d3ad6a3c-8841-4dd1-9309-91975b7ff7c8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" Jun 02 01:25:32.454299 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.454131 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" Jun 02 01:25:32.615893 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.615742 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt"] Jun 02 01:25:32.619931 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:32.619898 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ad6a3c_8841_4dd1_9309_91975b7ff7c8.slice/crio-573c300c38b73906be9ce0385b97440a0f0e13b2e503e658c5c364f0a122f54a WatchSource:0}: Error finding container 573c300c38b73906be9ce0385b97440a0f0e13b2e503e658c5c364f0a122f54a: Status 404 returned error can't find the container with id 573c300c38b73906be9ce0385b97440a0f0e13b2e503e658c5c364f0a122f54a Jun 02 01:25:32.816098 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.816018 2564 generic.go:358] "Generic (PLEG): container finished" podID="51bbd087-24d1-475d-ba30-bedf031a883c" containerID="bd7336f5c4f3d4f0cf07da15862692e75fef72ffb366b0f878b05d72abc91c0c" exitCode=0 Jun 02 01:25:32.816254 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.816113 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rrk44" event={"ID":"51bbd087-24d1-475d-ba30-bedf031a883c","Type":"ContainerDied","Data":"bd7336f5c4f3d4f0cf07da15862692e75fef72ffb366b0f878b05d72abc91c0c"} Jun 02 01:25:32.821102 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.821070 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" event={"ID":"61b4e7c0-a11b-4424-9664-a14c00c94b4e","Type":"ContainerStarted","Data":"ffef7844f756525ac4427ab6435d8146bf09084f033093498d7f682cd498f3cc"} Jun 02 01:25:32.821102 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.821107 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" event={"ID":"61b4e7c0-a11b-4424-9664-a14c00c94b4e","Type":"ContainerStarted","Data":"2a922aa68c626bf121653e68fb0cb8a7decfd1613af70998c094ef44cb67c658"} Jun 02 01:25:32.821102 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.821121 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" event={"ID":"61b4e7c0-a11b-4424-9664-a14c00c94b4e","Type":"ContainerStarted","Data":"3ebcaa232ce9b71e9e0c1d1ee1422b84184d5f711f094fff01da8c3f0f17d9bb"} Jun 02 01:25:32.823349 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.823323 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" event={"ID":"d3ad6a3c-8841-4dd1-9309-91975b7ff7c8","Type":"ContainerStarted","Data":"573c300c38b73906be9ce0385b97440a0f0e13b2e503e658c5c364f0a122f54a"} Jun 02 01:25:32.856178 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.854722 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-8gnlt" podStartSLOduration=4.6466678869999996 podStartE2EDuration="6.854705346s" podCreationTimestamp="2026-06-02 01:25:26 +0000 UTC" firstStartedPulling="2026-06-02 01:25:29.666562087 +0000 UTC m=+157.871022347" lastFinishedPulling="2026-06-02 01:25:31.87459954 +0000 UTC m=+160.079059806" observedRunningTime="2026-06-02 01:25:32.853953276 +0000 UTC m=+161.058413559" watchObservedRunningTime="2026-06-02 01:25:32.854705346 +0000 UTC m=+161.059165629" Jun 02 01:25:32.917791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.916423 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 01:25:32.943428 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.943379 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 01:25:32.943570 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.943555 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:32.947950 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.947884 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d4pq4\"" Jun 02 01:25:32.948841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948187 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 01:25:32.948841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948359 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 01:25:32.948841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948414 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 01:25:32.948841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948540 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-foqjmpqpq34j1\"" Jun 02 01:25:32.948841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948583 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 01:25:32.948841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948639 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 01:25:32.948841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948691 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 01:25:32.948841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948746 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 01:25:32.949291 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.948882 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 01:25:32.949291 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.949035 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 01:25:32.949396 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.949308 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 01:25:32.951991 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.951969 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 01:25:32.956133 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:32.956106 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 01:25:33.014040 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.013985 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014203 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014057 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvtzk\" (UniqueName: \"kubernetes.io/projected/d5c31886-d58e-4c6f-8efa-9b32271e7a75-kube-api-access-vvtzk\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014203 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014091 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c31886-d58e-4c6f-8efa-9b32271e7a75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014203 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014132 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014203 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014179 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-config\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014203 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014226 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014274 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014317 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014345 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c31886-d58e-4c6f-8efa-9b32271e7a75-config-out\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014372 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-web-config\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014799 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014513 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014799 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014560 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014799 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014799 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014695 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.014799 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.014717 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.115478 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115436 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.115688 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115490 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vvtzk\" (UniqueName: \"kubernetes.io/projected/d5c31886-d58e-4c6f-8efa-9b32271e7a75-kube-api-access-vvtzk\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.115688 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115518 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c31886-d58e-4c6f-8efa-9b32271e7a75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.115688 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115552 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.115688 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115586 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-config\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.115688 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115611 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.115688 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115634 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.115688 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115685 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115714 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115741 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115784 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c31886-d58e-4c6f-8efa-9b32271e7a75-config-out\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115809 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115835 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-web-config\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115872 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115906 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115962 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.115994 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.116432 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.116023 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.117195 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.116563 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.117195 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.116646 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.117195 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.116814 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.120587 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.120275 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.122026 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.121683 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c31886-d58e-4c6f-8efa-9b32271e7a75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.122026 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.121764 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.122026 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.121996 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.122679 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.122493 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-config\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.122805 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.122785 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c31886-d58e-4c6f-8efa-9b32271e7a75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.123373 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.123305 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.123799 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.123775 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-web-config\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.125174 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.124554 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c31886-d58e-4c6f-8efa-9b32271e7a75-config-out\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.125174 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.124909 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.125174 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.125132 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.126107 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.126068 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.126259 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.126218 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c31886-d58e-4c6f-8efa-9b32271e7a75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.126558 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.126518 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d5c31886-d58e-4c6f-8efa-9b32271e7a75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.127443 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.127391 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvtzk\" (UniqueName: \"kubernetes.io/projected/d5c31886-d58e-4c6f-8efa-9b32271e7a75-kube-api-access-vvtzk\") pod \"prometheus-k8s-0\" (UID: \"d5c31886-d58e-4c6f-8efa-9b32271e7a75\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:33.280669 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:33.280608 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:34.126937 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.126894 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:25:34.127318 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.127047 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:25:34.130279 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.130258 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a88a237-04e2-41a9-a1c5-dedfb5bc6df0-metrics-tls\") pod \"dns-default-tfpsh\" (UID: \"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0\") " pod="openshift-dns/dns-default-tfpsh" Jun 02 01:25:34.131146 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.131121 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99aff132-a1e2-4586-80cb-cee5ad8238bc-cert\") pod \"ingress-canary-gtr8q\" (UID: \"99aff132-a1e2-4586-80cb-cee5ad8238bc\") " pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:25:34.298261 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.298235 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xz25b\"" Jun 02 01:25:34.298426 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.298239 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8tqnd\"" Jun 02 01:25:34.305793 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.305697 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gtr8q" Jun 02 01:25:34.306463 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.306163 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tfpsh" Jun 02 01:25:34.399793 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.399766 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 01:25:34.541000 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.540897 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tfpsh"] Jun 02 01:25:34.553668 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.552873 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gtr8q"] Jun 02 01:25:34.560344 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:25:34.560314 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99aff132_a1e2_4586_80cb_cee5ad8238bc.slice/crio-4ee4926e5023967da724730bebe215b17b37c4731da6e8183cc80795321d190b WatchSource:0}: Error finding container 4ee4926e5023967da724730bebe215b17b37c4731da6e8183cc80795321d190b: Status 404 returned error can't find the container with id 4ee4926e5023967da724730bebe215b17b37c4731da6e8183cc80795321d190b Jun 02 01:25:34.831601 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.831544 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-675b845476-klc5n" event={"ID":"762b97d5-b17b-432b-be89-715419a3d70f","Type":"ContainerStarted","Data":"5490535d1500bc8859538e52fe90080e7674c12c80329dafc2a41516c5d728e8"} Jun 02 01:25:34.834281 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.834241 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" event={"ID":"91ac6667-5311-4e0a-93a1-dad9f25b6713","Type":"ContainerStarted","Data":"c291244448b13c4d894bd6c3e8b74aa1069c2177d6413d429f629d88e77de635"} Jun 02 01:25:34.834281 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.834282 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" event={"ID":"91ac6667-5311-4e0a-93a1-dad9f25b6713","Type":"ContainerStarted","Data":"9013be340be4b0deedfbc1351273c54d2b11b2439204f3cd1e2484bc51d28311"} Jun 02 01:25:34.834528 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.834300 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" event={"ID":"91ac6667-5311-4e0a-93a1-dad9f25b6713","Type":"ContainerStarted","Data":"3bd1e9dc167e9a5961ee1f37effb93c142311f4e84c295571b0e45c9affca35e"} Jun 02 01:25:34.836245 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.836203 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5c31886-d58e-4c6f-8efa-9b32271e7a75","Type":"ContainerStarted","Data":"e131c08bf4017fd245468c2593df524d7edf1294bc3c090d6708f756585cfb45"} Jun 02 01:25:34.837633 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.837596 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gtr8q" event={"ID":"99aff132-a1e2-4586-80cb-cee5ad8238bc","Type":"ContainerStarted","Data":"4ee4926e5023967da724730bebe215b17b37c4731da6e8183cc80795321d190b"} Jun 02 01:25:34.840732 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.840709 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rrk44" event={"ID":"51bbd087-24d1-475d-ba30-bedf031a883c","Type":"ContainerStarted","Data":"f177b779898033c7f991f6bbaf4efd0882b5e79d53bb22cf7478708daf067602"} Jun 02 01:25:34.840834 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.840739 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rrk44" event={"ID":"51bbd087-24d1-475d-ba30-bedf031a883c","Type":"ContainerStarted","Data":"d0368966d98a79f03a453df95cd4f911b822737ad1f91d495c715d9b38eed4b0"} Jun 02 01:25:34.842592 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.842560 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfpsh" event={"ID":"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0","Type":"ContainerStarted","Data":"af359c1de114965c18d874d6d75fcc2bc1b0b6b3d4e727bf58cee35a2710fc45"} Jun 02 01:25:34.850070 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.849999 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-675b845476-klc5n" podStartSLOduration=1.363139562 podStartE2EDuration="3.84998297s" podCreationTimestamp="2026-06-02 01:25:31 +0000 UTC" firstStartedPulling="2026-06-02 01:25:31.732440227 +0000 UTC m=+159.936900487" lastFinishedPulling="2026-06-02 01:25:34.219283633 +0000 UTC m=+162.423743895" observedRunningTime="2026-06-02 01:25:34.848114116 +0000 UTC m=+163.052574400" watchObservedRunningTime="2026-06-02 01:25:34.84998297 +0000 UTC m=+163.054443253" Jun 02 01:25:34.873078 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:34.872963 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rrk44" podStartSLOduration=6.848635636 podStartE2EDuration="8.872946631s" podCreationTimestamp="2026-06-02 01:25:26 +0000 UTC" firstStartedPulling="2026-06-02 01:25:29.531450473 +0000 UTC m=+157.735910738" lastFinishedPulling="2026-06-02 01:25:31.555761456 +0000 UTC m=+159.760221733" observedRunningTime="2026-06-02 01:25:34.870511674 +0000 UTC m=+163.074971985" watchObservedRunningTime="2026-06-02 01:25:34.872946631 +0000 UTC m=+163.077406915" Jun 02 01:25:35.031516 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.031328 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" podUID="ed261979-0c4f-4322-a186-adffe60bffcb" containerName="registry" containerID="cri-o://445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8" gracePeriod=30 Jun 02 01:25:35.063644 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:25:35.063585 2564 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded261979_0c4f_4322_a186_adffe60bffcb.slice/crio-445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8.scope\": RecentStats: unable to find data in memory cache]" Jun 02 01:25:35.567343 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.567316 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:25:35.656212 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656122 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed261979-0c4f-4322-a186-adffe60bffcb-ca-trust-extracted\") pod \"ed261979-0c4f-4322-a186-adffe60bffcb\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " Jun 02 01:25:35.656212 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656166 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-bound-sa-token\") pod \"ed261979-0c4f-4322-a186-adffe60bffcb\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " Jun 02 01:25:35.656212 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656208 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-registry-certificates\") pod \"ed261979-0c4f-4322-a186-adffe60bffcb\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " Jun 02 01:25:35.656510 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656236 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-image-registry-private-configuration\") pod \"ed261979-0c4f-4322-a186-adffe60bffcb\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " Jun 02 01:25:35.656510 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656267 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bnb4\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-kube-api-access-6bnb4\") pod \"ed261979-0c4f-4322-a186-adffe60bffcb\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " Jun 02 01:25:35.656510 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656318 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") pod \"ed261979-0c4f-4322-a186-adffe60bffcb\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " Jun 02 01:25:35.656510 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656357 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-trusted-ca\") pod \"ed261979-0c4f-4322-a186-adffe60bffcb\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " Jun 02 01:25:35.656510 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656399 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-installation-pull-secrets\") pod \"ed261979-0c4f-4322-a186-adffe60bffcb\" (UID: \"ed261979-0c4f-4322-a186-adffe60bffcb\") " Jun 02 01:25:35.656804 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656637 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ed261979-0c4f-4322-a186-adffe60bffcb" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 01:25:35.657023 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.656989 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ed261979-0c4f-4322-a186-adffe60bffcb" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 01:25:35.659108 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.659072 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ed261979-0c4f-4322-a186-adffe60bffcb" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 01:25:35.659217 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.659176 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ed261979-0c4f-4322-a186-adffe60bffcb" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 01:25:35.659383 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.659365 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ed261979-0c4f-4322-a186-adffe60bffcb" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 01:25:35.659860 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.659830 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-kube-api-access-6bnb4" (OuterVolumeSpecName: "kube-api-access-6bnb4") pod "ed261979-0c4f-4322-a186-adffe60bffcb" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb"). InnerVolumeSpecName "kube-api-access-6bnb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 01:25:35.660227 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.660200 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ed261979-0c4f-4322-a186-adffe60bffcb" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 01:25:35.669054 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.667747 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed261979-0c4f-4322-a186-adffe60bffcb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ed261979-0c4f-4322-a186-adffe60bffcb" (UID: "ed261979-0c4f-4322-a186-adffe60bffcb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 01:25:35.757553 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.757511 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-registry-tls\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:25:35.757553 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.757548 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-trusted-ca\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:25:35.757839 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.757563 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-installation-pull-secrets\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:25:35.757839 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.757580 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed261979-0c4f-4322-a186-adffe60bffcb-ca-trust-extracted\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:25:35.757839 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.757594 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-bound-sa-token\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:25:35.757839 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.757609 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed261979-0c4f-4322-a186-adffe60bffcb-registry-certificates\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:25:35.757839 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.757625 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed261979-0c4f-4322-a186-adffe60bffcb-image-registry-private-configuration\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:25:35.757839 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.757640 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bnb4\" (UniqueName: \"kubernetes.io/projected/ed261979-0c4f-4322-a186-adffe60bffcb-kube-api-access-6bnb4\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:25:35.853672 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.850262 2564 generic.go:358] "Generic (PLEG): container finished" podID="ed261979-0c4f-4322-a186-adffe60bffcb" containerID="445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8" exitCode=0 Jun 02 01:25:35.853672 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.851173 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" Jun 02 01:25:35.854325 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.853980 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" event={"ID":"ed261979-0c4f-4322-a186-adffe60bffcb","Type":"ContainerDied","Data":"445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8"} Jun 02 01:25:35.854325 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.854029 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9d94ccc68-rtxfx" event={"ID":"ed261979-0c4f-4322-a186-adffe60bffcb","Type":"ContainerDied","Data":"a9133499a96545921da2826280426c904ccc60f5979fd719bd9a68ab59707c66"} Jun 02 01:25:35.854325 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.854050 2564 scope.go:117] "RemoveContainer" containerID="445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8" Jun 02 01:25:35.886123 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.885878 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9d94ccc68-rtxfx"] Jun 02 01:25:35.894108 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:35.894055 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9d94ccc68-rtxfx"] Jun 02 01:25:36.381107 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:36.380861 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed261979-0c4f-4322-a186-adffe60bffcb" path="/var/lib/kubelet/pods/ed261979-0c4f-4322-a186-adffe60bffcb/volumes" Jun 02 01:25:36.384988 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:36.384890 2564 scope.go:117] "RemoveContainer" containerID="445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8" Jun 02 01:25:36.385541 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:25:36.385449 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8\": container with ID starting with 445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8 not found: ID does not exist" containerID="445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8" Jun 02 01:25:36.385541 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:36.385493 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8"} err="failed to get container status \"445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8\": rpc error: code = NotFound desc = could not find container \"445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8\": container with ID starting with 445861add9669d3c7e23d6901734f0ce52775e1fff3a7fb21b96a2c21e2fe8b8 not found: ID does not exist" Jun 02 01:25:38.864338 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.864302 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfpsh" event={"ID":"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0","Type":"ContainerStarted","Data":"80e397d55199128956a08ae1857c59dd53592c4902de567b20609a8428c74048"} Jun 02 01:25:38.867726 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.867644 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" event={"ID":"91ac6667-5311-4e0a-93a1-dad9f25b6713","Type":"ContainerStarted","Data":"d230dab4bc6a9025140680001ef024eb8caec717be99e8c0df5ecc42aacab408"} Jun 02 01:25:38.867726 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.867707 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" event={"ID":"91ac6667-5311-4e0a-93a1-dad9f25b6713","Type":"ContainerStarted","Data":"5fe08ed6c6810999b790430f5ebe4f9730c96456b85bc96d369048d6a52d4855"} Jun 02 01:25:38.870073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.869504 2564 generic.go:358] "Generic (PLEG): container finished" podID="d5c31886-d58e-4c6f-8efa-9b32271e7a75" containerID="82cd5e837208d96918cca9afb0ecc3f173def01ca449f975d2af33f566c7666e" exitCode=0 Jun 02 01:25:38.870073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.869582 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5c31886-d58e-4c6f-8efa-9b32271e7a75","Type":"ContainerDied","Data":"82cd5e837208d96918cca9afb0ecc3f173def01ca449f975d2af33f566c7666e"} Jun 02 01:25:38.871935 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.871912 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" event={"ID":"d3ad6a3c-8841-4dd1-9309-91975b7ff7c8","Type":"ContainerStarted","Data":"1d9712542373fd693a751b20d6f13b1f98aed1a2a9329d1b7d39bf4c45aaaf48"} Jun 02 01:25:38.872532 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.872510 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" Jun 02 01:25:38.879365 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.879337 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" Jun 02 01:25:38.917073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:38.917030 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wnwvt" podStartSLOduration=3.158879716 podStartE2EDuration="7.917010288s" podCreationTimestamp="2026-06-02 01:25:31 +0000 UTC" firstStartedPulling="2026-06-02 01:25:32.622536835 +0000 UTC m=+160.826997112" lastFinishedPulling="2026-06-02 01:25:37.380649437 +0000 UTC m=+165.585127684" observedRunningTime="2026-06-02 01:25:38.915508957 +0000 UTC m=+167.119969239" watchObservedRunningTime="2026-06-02 01:25:38.917010288 +0000 UTC m=+167.121470570" Jun 02 01:25:39.877749 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:39.877705 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" event={"ID":"91ac6667-5311-4e0a-93a1-dad9f25b6713","Type":"ContainerStarted","Data":"b0b69794f6d99b5c818e2f64732f7f861c6de3b7d7e0c3a1aabb26963ec50206"} Jun 02 01:25:39.878303 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:39.878027 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:39.879609 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:39.879573 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gtr8q" event={"ID":"99aff132-a1e2-4586-80cb-cee5ad8238bc","Type":"ContainerStarted","Data":"31c6c90effffd6079492b43fe8aa0e9951f58166ca5928f7740aee94dd145843"} Jun 02 01:25:39.882381 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:39.882345 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfpsh" event={"ID":"0a88a237-04e2-41a9-a1c5-dedfb5bc6df0","Type":"ContainerStarted","Data":"0252ac3f286b932cb39a704ee1c8a2350d3ca4b222a9b9bda3ae35364ce65630"} Jun 02 01:25:39.885892 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:39.885871 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" Jun 02 01:25:39.900506 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:39.900467 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78987677b8-nptt4" podStartSLOduration=2.634908926 podStartE2EDuration="10.900455323s" podCreationTimestamp="2026-06-02 01:25:29 +0000 UTC" firstStartedPulling="2026-06-02 01:25:30.194693148 +0000 UTC m=+158.399153408" lastFinishedPulling="2026-06-02 01:25:38.460239537 +0000 UTC m=+166.664699805" observedRunningTime="2026-06-02 01:25:39.900120361 +0000 UTC m=+168.104580678" watchObservedRunningTime="2026-06-02 01:25:39.900455323 +0000 UTC m=+168.104915615" Jun 02 01:25:39.917304 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:39.917263 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-tfpsh" podStartSLOduration=130.012251505 podStartE2EDuration="2m13.917252117s" podCreationTimestamp="2026-06-02 01:23:26 +0000 UTC" firstStartedPulling="2026-06-02 01:25:34.559305077 +0000 UTC m=+162.763765342" lastFinishedPulling="2026-06-02 01:25:38.464305683 +0000 UTC m=+166.668765954" observedRunningTime="2026-06-02 01:25:39.915884627 +0000 UTC m=+168.120344912" watchObservedRunningTime="2026-06-02 01:25:39.917252117 +0000 UTC m=+168.121712398" Jun 02 01:25:39.953732 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:39.953687 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gtr8q" podStartSLOduration=129.735467008 podStartE2EDuration="2m13.953673912s" podCreationTimestamp="2026-06-02 01:23:26 +0000 UTC" firstStartedPulling="2026-06-02 01:25:34.564346236 +0000 UTC m=+162.768806503" lastFinishedPulling="2026-06-02 01:25:38.782553137 +0000 UTC m=+166.987013407" observedRunningTime="2026-06-02 01:25:39.952041666 +0000 UTC m=+168.156501951" watchObservedRunningTime="2026-06-02 01:25:39.953673912 +0000 UTC m=+168.158134191" Jun 02 01:25:40.361442 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:40.361409 2564 patch_prober.go:28] interesting pod/image-registry-7f8755f774-lbkfj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 01:25:40.361593 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:40.361462 2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" podUID="0362e1c2-f61f-4871-a49a-348ad82648e6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 01:25:40.886408 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:40.886327 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tfpsh" Jun 02 01:25:41.736956 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:41.736280 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f8755f774-lbkfj" Jun 02 01:25:42.894275 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:42.894245 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5c31886-d58e-4c6f-8efa-9b32271e7a75","Type":"ContainerStarted","Data":"2a31634297a1c7c396ee370bc0492d8b9dec409f242e3055573d7618a33276a1"} Jun 02 01:25:42.894612 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:42.894279 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5c31886-d58e-4c6f-8efa-9b32271e7a75","Type":"ContainerStarted","Data":"ef777de9d8b2e2bb69e7ccb28d3c836875dcb2657ca5c0f4bc2573d34ecedb45"} Jun 02 01:25:43.904256 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:43.904216 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5c31886-d58e-4c6f-8efa-9b32271e7a75","Type":"ContainerStarted","Data":"c8c4e6f58b775b2c8e2a2fd7bbe5e81407744243bb878926372e4e60bc0c7676"} Jun 02 01:25:43.904256 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:43.904258 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5c31886-d58e-4c6f-8efa-9b32271e7a75","Type":"ContainerStarted","Data":"eb17999feadce39d73a800a0645df0ace4289a577782469bf3481669d714915b"} Jun 02 01:25:43.904763 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:43.904273 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5c31886-d58e-4c6f-8efa-9b32271e7a75","Type":"ContainerStarted","Data":"7bab883d6b714eb594af929f2f51f7f221d174788314c58f4d640b336614ae82"} Jun 02 01:25:43.904763 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:43.904285 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5c31886-d58e-4c6f-8efa-9b32271e7a75","Type":"ContainerStarted","Data":"6d1008909c85e26fc98acab8ecae276b4fe3013a88aa28b2c98ef62fb608e546"} Jun 02 01:25:43.930706 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:43.930613 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.785334341 podStartE2EDuration="11.930595449s" podCreationTimestamp="2026-06-02 01:25:32 +0000 UTC" firstStartedPulling="2026-06-02 01:25:34.417928951 +0000 UTC m=+162.622389215" lastFinishedPulling="2026-06-02 01:25:42.56319006 +0000 UTC m=+170.767650323" observedRunningTime="2026-06-02 01:25:43.928965904 +0000 UTC m=+172.133426188" watchObservedRunningTime="2026-06-02 01:25:43.930595449 +0000 UTC m=+172.135055731" Jun 02 01:25:48.281732 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:48.281696 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:25:50.891492 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:50.891460 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tfpsh" Jun 02 01:25:51.370874 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:51.370837 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:25:51.370874 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:25:51.370877 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:26:01.496022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:01.495989 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-69b6cbdb88-q5v5z_3e5e657c-d1d2-4183-aae9-86ede8ed3897/cluster-monitoring-operator/0.log" Jun 02 01:26:01.694424 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:01.694393 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-8gnlt_61b4e7c0-a11b-4424-9664-a14c00c94b4e/kube-state-metrics/0.log" Jun 02 01:26:01.894082 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:01.894054 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-8gnlt_61b4e7c0-a11b-4424-9664-a14c00c94b4e/kube-rbac-proxy-main/0.log" Jun 02 01:26:02.094047 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:02.094017 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-8gnlt_61b4e7c0-a11b-4424-9664-a14c00c94b4e/kube-rbac-proxy-self/0.log" Jun 02 01:26:02.293738 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:02.293640 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-675b845476-klc5n_762b97d5-b17b-432b-be89-715419a3d70f/metrics-server/0.log" Jun 02 01:26:02.493761 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:02.493733 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-wnwvt_d3ad6a3c-8841-4dd1-9309-91975b7ff7c8/monitoring-plugin/0.log" Jun 02 01:26:03.897915 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:03.897886 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rrk44_51bbd087-24d1-475d-ba30-bedf031a883c/init-textfile/0.log" Jun 02 01:26:04.098861 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:04.098827 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rrk44_51bbd087-24d1-475d-ba30-bedf031a883c/node-exporter/0.log" Jun 02 01:26:04.294880 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:04.294797 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rrk44_51bbd087-24d1-475d-ba30-bedf031a883c/kube-rbac-proxy/0.log" Jun 02 01:26:05.094243 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:05.094214 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/init-config-reloader/0.log" Jun 02 01:26:05.295242 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:05.295213 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/prometheus/0.log" Jun 02 01:26:05.493857 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:05.493833 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/config-reloader/0.log" Jun 02 01:26:05.694921 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:05.694892 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/thanos-sidecar/0.log" Jun 02 01:26:05.894215 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:05.894179 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/kube-rbac-proxy-web/0.log" Jun 02 01:26:06.094309 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:06.094279 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/kube-rbac-proxy/0.log" Jun 02 01:26:06.294906 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:06.294829 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/kube-rbac-proxy-thanos/0.log" Jun 02 01:26:07.094277 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:07.094246 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/thanos-query/0.log" Jun 02 01:26:07.294063 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:07.294032 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/kube-rbac-proxy-web/0.log" Jun 02 01:26:07.496841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:07.496811 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/kube-rbac-proxy/0.log" Jun 02 01:26:07.694171 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:07.694138 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/prom-label-proxy/0.log" Jun 02 01:26:07.894126 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:07.894091 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/kube-rbac-proxy-rules/0.log" Jun 02 01:26:08.094902 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:08.094877 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/kube-rbac-proxy-metrics/0.log" Jun 02 01:26:08.294122 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:08.294045 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-qm5j8_fb11ade2-f6f5-4fea-bf71-8193a5071c03/networking-console-plugin/0.log" Jun 02 01:26:09.295423 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:09.295397 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-zsj96_097f5758-d818-44c7-bd95-39a8e63b75e0/download-server/0.log" Jun 02 01:26:09.495456 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:09.495414 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c7cfdd785-nlm5p_16360c44-5de0-4680-99d0-0b9e9120ec63/router/0.log" Jun 02 01:26:09.694008 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:09.693977 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gtr8q_99aff132-a1e2-4586-80cb-cee5ad8238bc/serve-healthcheck-canary/0.log" Jun 02 01:26:11.377436 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:11.377406 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:26:11.381997 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:11.381962 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-675b845476-klc5n" Jun 02 01:26:33.281860 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:33.281804 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:26:33.298331 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:33.298295 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:26:34.077999 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:26:34.077971 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 01:27:52.292314 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:27:52.292287 2564 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 01:29:11.610600 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.610556 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts"] Jun 02 01:29:11.611086 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.610953 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed261979-0c4f-4322-a186-adffe60bffcb" containerName="registry" Jun 02 01:29:11.611086 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.610966 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed261979-0c4f-4322-a186-adffe60bffcb" containerName="registry" Jun 02 01:29:11.611086 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.611014 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="ed261979-0c4f-4322-a186-adffe60bffcb" containerName="registry" Jun 02 01:29:11.613001 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.612985 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.617933 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.617907 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 01:29:11.618093 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.617962 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 01:29:11.618682 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.618474 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 01:29:11.618801 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.618730 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-56xw8\"" Jun 02 01:29:11.618901 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.618878 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 01:29:11.625339 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.625311 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts"] Jun 02 01:29:11.739512 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.739475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/830114f5-92cf-4d17-81fe-cf1e7efcf37c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.739725 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.739528 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/830114f5-92cf-4d17-81fe-cf1e7efcf37c-manifests\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.739725 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.739627 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/830114f5-92cf-4d17-81fe-cf1e7efcf37c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.739725 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.739678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/830114f5-92cf-4d17-81fe-cf1e7efcf37c-tmp\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.739725 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.739713 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mztn6\" (UniqueName: \"kubernetes.io/projected/830114f5-92cf-4d17-81fe-cf1e7efcf37c-kube-api-access-mztn6\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.840783 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.840736 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/830114f5-92cf-4d17-81fe-cf1e7efcf37c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.840783 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.840782 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/830114f5-92cf-4d17-81fe-cf1e7efcf37c-tmp\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.841039 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.840808 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mztn6\" (UniqueName: \"kubernetes.io/projected/830114f5-92cf-4d17-81fe-cf1e7efcf37c-kube-api-access-mztn6\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.841039 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.840943 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/830114f5-92cf-4d17-81fe-cf1e7efcf37c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.841039 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.840994 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/830114f5-92cf-4d17-81fe-cf1e7efcf37c-manifests\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.841368 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.841343 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/830114f5-92cf-4d17-81fe-cf1e7efcf37c-manifests\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.843177 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.843156 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/830114f5-92cf-4d17-81fe-cf1e7efcf37c-tmp\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.843316 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.843300 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/830114f5-92cf-4d17-81fe-cf1e7efcf37c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.843471 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.843450 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/830114f5-92cf-4d17-81fe-cf1e7efcf37c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.854716 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.854685 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mztn6\" (UniqueName: \"kubernetes.io/projected/830114f5-92cf-4d17-81fe-cf1e7efcf37c-kube-api-access-mztn6\") pod \"opendatahub-operator-controller-manager-6758f7c588-dxhts\" (UID: \"830114f5-92cf-4d17-81fe-cf1e7efcf37c\") " pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:11.924070 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:11.923980 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:12.063190 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:12.063164 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts"] Jun 02 01:29:12.065342 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:29:12.065310 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830114f5_92cf_4d17_81fe_cf1e7efcf37c.slice/crio-e13b82d6809d680c671e0e3cf93ab2a227589fa75d2a5675631964bdf2151b97 WatchSource:0}: Error finding container e13b82d6809d680c671e0e3cf93ab2a227589fa75d2a5675631964bdf2151b97: Status 404 returned error can't find the container with id e13b82d6809d680c671e0e3cf93ab2a227589fa75d2a5675631964bdf2151b97 Jun 02 01:29:12.066880 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:12.066863 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 01:29:12.521465 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:12.521426 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" event={"ID":"830114f5-92cf-4d17-81fe-cf1e7efcf37c","Type":"ContainerStarted","Data":"e13b82d6809d680c671e0e3cf93ab2a227589fa75d2a5675631964bdf2151b97"} Jun 02 01:29:16.544271 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:16.544236 2564 generic.go:358] "Generic (PLEG): container finished" podID="830114f5-92cf-4d17-81fe-cf1e7efcf37c" containerID="8234e7fd54391b5d9a7f29559d41236842724a6c95b14795b10bf3d5c3598f60" exitCode=0 Jun 02 01:29:16.544668 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:16.544277 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" event={"ID":"830114f5-92cf-4d17-81fe-cf1e7efcf37c","Type":"ContainerDied","Data":"8234e7fd54391b5d9a7f29559d41236842724a6c95b14795b10bf3d5c3598f60"} Jun 02 01:29:17.550346 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:17.550304 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" event={"ID":"830114f5-92cf-4d17-81fe-cf1e7efcf37c","Type":"ContainerStarted","Data":"0bb0fce1dab9144be5fd1045c97a998f5045e26397a8a5d6d10fed8a7980f905"} Jun 02 01:29:17.550877 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:17.550421 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:17.573600 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:17.573548 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" podStartSLOduration=1.630199283 podStartE2EDuration="6.573532484s" podCreationTimestamp="2026-06-02 01:29:11 +0000 UTC" firstStartedPulling="2026-06-02 01:29:12.066981377 +0000 UTC m=+380.271441637" lastFinishedPulling="2026-06-02 01:29:17.010314559 +0000 UTC m=+385.214774838" observedRunningTime="2026-06-02 01:29:17.57155664 +0000 UTC m=+385.776016915" watchObservedRunningTime="2026-06-02 01:29:17.573532484 +0000 UTC m=+385.777992765" Jun 02 01:29:28.555528 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:28.555492 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6758f7c588-dxhts" Jun 02 01:29:31.919110 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.919069 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr"] Jun 02 01:29:31.930414 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.930383 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:31.932149 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.932120 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr"] Jun 02 01:29:31.933864 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.933843 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 01:29:31.933980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.933890 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 01:29:31.933980 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.933843 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 01:29:31.934089 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.933850 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 01:29:31.934089 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.933849 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2rm82\"" Jun 02 01:29:31.934089 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:31.933897 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 01:29:32.009359 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.009320 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-cert\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.009359 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.009359 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-manager-config\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.009568 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.009462 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnjcm\" (UniqueName: \"kubernetes.io/projected/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-kube-api-access-qnjcm\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.009568 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.009501 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-metrics-cert\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.109987 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.109949 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qnjcm\" (UniqueName: \"kubernetes.io/projected/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-kube-api-access-qnjcm\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.109987 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.109995 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-metrics-cert\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.110246 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.110036 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-cert\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.110246 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.110058 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-manager-config\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.110667 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.110630 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-manager-config\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.112579 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.112560 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-metrics-cert\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.112641 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.112601 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-cert\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.121042 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.121013 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnjcm\" (UniqueName: \"kubernetes.io/projected/29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7-kube-api-access-qnjcm\") pod \"lws-controller-manager-6df7964bff-gqrnr\" (UID: \"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7\") " pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.240905 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.240809 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:32.376831 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:29:32.376797 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b5c3aa_0f07_4ccf_b9e8_c855e71b65d7.slice/crio-db4e2af15bcf491a12117239f7aaeab90566d6a23f035e3e58a65c809bc23e2c WatchSource:0}: Error finding container db4e2af15bcf491a12117239f7aaeab90566d6a23f035e3e58a65c809bc23e2c: Status 404 returned error can't find the container with id db4e2af15bcf491a12117239f7aaeab90566d6a23f035e3e58a65c809bc23e2c Jun 02 01:29:32.378293 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.378271 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr"] Jun 02 01:29:32.601440 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:32.601351 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" event={"ID":"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7","Type":"ContainerStarted","Data":"db4e2af15bcf491a12117239f7aaeab90566d6a23f035e3e58a65c809bc23e2c"} Jun 02 01:29:34.248161 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.248127 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-865b6fd47-znw7r"] Jun 02 01:29:34.251544 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.251515 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.254251 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.254226 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 01:29:34.254387 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.254232 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 01:29:34.254387 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.254233 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tvkjx\"" Jun 02 01:29:34.267390 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.266916 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-865b6fd47-znw7r"] Jun 02 01:29:34.332821 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.332780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84057c87-4bac-4372-8add-f7bd2ece8060-tmp\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.332993 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.332835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z548k\" (UniqueName: \"kubernetes.io/projected/84057c87-4bac-4372-8add-f7bd2ece8060-kube-api-access-z548k\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.332993 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.332885 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84057c87-4bac-4372-8add-f7bd2ece8060-tls-certs\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.434339 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.434297 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84057c87-4bac-4372-8add-f7bd2ece8060-tls-certs\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.434537 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.434448 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84057c87-4bac-4372-8add-f7bd2ece8060-tmp\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.434537 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.434497 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z548k\" (UniqueName: \"kubernetes.io/projected/84057c87-4bac-4372-8add-f7bd2ece8060-kube-api-access-z548k\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.434537 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:29:34.434502 2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 02 01:29:34.434808 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:29:34.434602 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84057c87-4bac-4372-8add-f7bd2ece8060-tls-certs podName:84057c87-4bac-4372-8add-f7bd2ece8060 nodeName:}" failed. No retries permitted until 2026-06-02 01:29:34.934576697 +0000 UTC m=+403.139036973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84057c87-4bac-4372-8add-f7bd2ece8060-tls-certs") pod "kube-auth-proxy-865b6fd47-znw7r" (UID: "84057c87-4bac-4372-8add-f7bd2ece8060") : secret "kube-auth-proxy-tls" not found Jun 02 01:29:34.437323 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.437268 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84057c87-4bac-4372-8add-f7bd2ece8060-tmp\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.447476 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.447444 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z548k\" (UniqueName: \"kubernetes.io/projected/84057c87-4bac-4372-8add-f7bd2ece8060-kube-api-access-z548k\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.939904 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.939863 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84057c87-4bac-4372-8add-f7bd2ece8060-tls-certs\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:34.942496 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:34.942460 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84057c87-4bac-4372-8add-f7bd2ece8060-tls-certs\") pod \"kube-auth-proxy-865b6fd47-znw7r\" (UID: \"84057c87-4bac-4372-8add-f7bd2ece8060\") " pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:35.163894 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:35.163848 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" Jun 02 01:29:35.516616 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:35.516582 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-865b6fd47-znw7r"] Jun 02 01:29:35.520253 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:29:35.520205 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84057c87_4bac_4372_8add_f7bd2ece8060.slice/crio-61afc34cf62f6a459c5d3260f6b5b7e17df88ac46796b237ab87e82a46921971 WatchSource:0}: Error finding container 61afc34cf62f6a459c5d3260f6b5b7e17df88ac46796b237ab87e82a46921971: Status 404 returned error can't find the container with id 61afc34cf62f6a459c5d3260f6b5b7e17df88ac46796b237ab87e82a46921971 Jun 02 01:29:35.614019 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:35.613981 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" event={"ID":"29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7","Type":"ContainerStarted","Data":"96b4ec9de5ad6996a634c07046cd1384f944a905cccabb45027c053999f0c692"} Jun 02 01:29:35.614206 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:35.614113 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:29:35.615113 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:35.615087 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" event={"ID":"84057c87-4bac-4372-8add-f7bd2ece8060","Type":"ContainerStarted","Data":"61afc34cf62f6a459c5d3260f6b5b7e17df88ac46796b237ab87e82a46921971"} Jun 02 01:29:35.631313 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:35.631202 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" podStartSLOduration=1.566533408 podStartE2EDuration="4.631184212s" podCreationTimestamp="2026-06-02 01:29:31 +0000 UTC" firstStartedPulling="2026-06-02 01:29:32.378823994 +0000 UTC m=+400.583284258" lastFinishedPulling="2026-06-02 01:29:35.443474774 +0000 UTC m=+403.647935062" observedRunningTime="2026-06-02 01:29:35.63009414 +0000 UTC m=+403.834554445" watchObservedRunningTime="2026-06-02 01:29:35.631184212 +0000 UTC m=+403.835644494" Jun 02 01:29:38.631178 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:38.631134 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" event={"ID":"84057c87-4bac-4372-8add-f7bd2ece8060","Type":"ContainerStarted","Data":"c44fa23bdec2e05c0d6d931be507fe1e745e8cc4ac6a597c15272785657f4bda"} Jun 02 01:29:38.649468 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:38.649418 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-865b6fd47-znw7r" podStartSLOduration=1.923158851 podStartE2EDuration="4.649403233s" podCreationTimestamp="2026-06-02 01:29:34 +0000 UTC" firstStartedPulling="2026-06-02 01:29:35.525891416 +0000 UTC m=+403.730351681" lastFinishedPulling="2026-06-02 01:29:38.252135796 +0000 UTC m=+406.456596063" observedRunningTime="2026-06-02 01:29:38.648407603 +0000 UTC m=+406.852867923" watchObservedRunningTime="2026-06-02 01:29:38.649403233 +0000 UTC m=+406.853863515" Jun 02 01:29:46.622025 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:29:46.621988 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6df7964bff-gqrnr" Jun 02 01:31:19.015928 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.015885 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w"] Jun 02 01:31:19.019009 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.018990 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.021947 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.021920 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 02 01:31:19.022075 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.022052 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 02 01:31:19.022147 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.022136 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gvlrz\"" Jun 02 01:31:19.022741 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.022720 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 01:31:19.022835 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.022744 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 01:31:19.035876 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.035848 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w"] Jun 02 01:31:19.135414 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.135367 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwsqp\" (UniqueName: \"kubernetes.io/projected/0f96b4b5-9080-4e65-8b35-220895896adc-kube-api-access-qwsqp\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.135414 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.135415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f96b4b5-9080-4e65-8b35-220895896adc-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.135630 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.135440 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0f96b4b5-9080-4e65-8b35-220895896adc-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.236636 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.236597 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f96b4b5-9080-4e65-8b35-220895896adc-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.236801 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.236644 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0f96b4b5-9080-4e65-8b35-220895896adc-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.236801 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.236743 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qwsqp\" (UniqueName: \"kubernetes.io/projected/0f96b4b5-9080-4e65-8b35-220895896adc-kube-api-access-qwsqp\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.236801 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:31:19.236750 2564 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 02 01:31:19.236936 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:31:19.236817 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f96b4b5-9080-4e65-8b35-220895896adc-plugin-serving-cert podName:0f96b4b5-9080-4e65-8b35-220895896adc nodeName:}" failed. No retries permitted until 2026-06-02 01:31:19.736800738 +0000 UTC m=+507.941260998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/0f96b4b5-9080-4e65-8b35-220895896adc-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-bv42w" (UID: "0f96b4b5-9080-4e65-8b35-220895896adc") : secret "plugin-serving-cert" not found Jun 02 01:31:19.237331 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.237314 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0f96b4b5-9080-4e65-8b35-220895896adc-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.251751 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.251719 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwsqp\" (UniqueName: \"kubernetes.io/projected/0f96b4b5-9080-4e65-8b35-220895896adc-kube-api-access-qwsqp\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.741643 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.741599 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f96b4b5-9080-4e65-8b35-220895896adc-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.744132 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.744102 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f96b4b5-9080-4e65-8b35-220895896adc-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-bv42w\" (UID: \"0f96b4b5-9080-4e65-8b35-220895896adc\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:19.935862 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:19.935827 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" Jun 02 01:31:20.071696 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:20.071671 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w"] Jun 02 01:31:20.074443 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:31:20.074416 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f96b4b5_9080_4e65_8b35_220895896adc.slice/crio-a1dfac1cd1feacb9cce692afcc77930b315f332cd81190c28854bbbc51cd6964 WatchSource:0}: Error finding container a1dfac1cd1feacb9cce692afcc77930b315f332cd81190c28854bbbc51cd6964: Status 404 returned error can't find the container with id a1dfac1cd1feacb9cce692afcc77930b315f332cd81190c28854bbbc51cd6964 Jun 02 01:31:20.963120 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:20.963080 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" event={"ID":"0f96b4b5-9080-4e65-8b35-220895896adc","Type":"ContainerStarted","Data":"a1dfac1cd1feacb9cce692afcc77930b315f332cd81190c28854bbbc51cd6964"} Jun 02 01:31:45.053480 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:45.053384 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" event={"ID":"0f96b4b5-9080-4e65-8b35-220895896adc","Type":"ContainerStarted","Data":"bf886d233bae42f6b96db76876484725543a8e7f7baab76431dc314e9375d65e"} Jun 02 01:31:45.080144 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:31:45.080081 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bv42w" podStartSLOduration=2.4706720349999998 podStartE2EDuration="27.080065983s" podCreationTimestamp="2026-06-02 01:31:18 +0000 UTC" firstStartedPulling="2026-06-02 01:31:20.075744578 +0000 UTC m=+508.280204857" lastFinishedPulling="2026-06-02 01:31:44.685138544 +0000 UTC m=+532.889598805" observedRunningTime="2026-06-02 01:31:45.079648387 +0000 UTC m=+533.284108669" watchObservedRunningTime="2026-06-02 01:31:45.080065983 +0000 UTC m=+533.284526305" Jun 02 01:32:18.546887 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.546845 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:32:18.610804 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.610765 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:32:18.610804 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.610811 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:32:18.611011 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.610931 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:18.613227 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.613208 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 02 01:32:18.661027 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.660987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c710341-1d2f-44c4-90f4-c86b3e3c2ed1-config-file\") pod \"limitador-limitador-78c99df468-9sm5p\" (UID: \"4c710341-1d2f-44c4-90f4-c86b3e3c2ed1\") " pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:18.661206 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.661080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68sl2\" (UniqueName: \"kubernetes.io/projected/4c710341-1d2f-44c4-90f4-c86b3e3c2ed1-kube-api-access-68sl2\") pod \"limitador-limitador-78c99df468-9sm5p\" (UID: \"4c710341-1d2f-44c4-90f4-c86b3e3c2ed1\") " pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:18.761824 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.761782 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c710341-1d2f-44c4-90f4-c86b3e3c2ed1-config-file\") pod \"limitador-limitador-78c99df468-9sm5p\" (UID: \"4c710341-1d2f-44c4-90f4-c86b3e3c2ed1\") " pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:18.762022 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.761879 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68sl2\" (UniqueName: \"kubernetes.io/projected/4c710341-1d2f-44c4-90f4-c86b3e3c2ed1-kube-api-access-68sl2\") pod \"limitador-limitador-78c99df468-9sm5p\" (UID: \"4c710341-1d2f-44c4-90f4-c86b3e3c2ed1\") " pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:18.762419 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.762398 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c710341-1d2f-44c4-90f4-c86b3e3c2ed1-config-file\") pod \"limitador-limitador-78c99df468-9sm5p\" (UID: \"4c710341-1d2f-44c4-90f4-c86b3e3c2ed1\") " pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:18.769351 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.769320 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68sl2\" (UniqueName: \"kubernetes.io/projected/4c710341-1d2f-44c4-90f4-c86b3e3c2ed1-kube-api-access-68sl2\") pod \"limitador-limitador-78c99df468-9sm5p\" (UID: \"4c710341-1d2f-44c4-90f4-c86b3e3c2ed1\") " pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:18.921853 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:18.921812 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:19.059799 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.059743 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:32:19.062552 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:32:19.062517 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c710341_1d2f_44c4_90f4_c86b3e3c2ed1.slice/crio-9682e3aa475770650238c73e99f0a0dbd5380c25aa9409fb763073f506ac050d WatchSource:0}: Error finding container 9682e3aa475770650238c73e99f0a0dbd5380c25aa9409fb763073f506ac050d: Status 404 returned error can't find the container with id 9682e3aa475770650238c73e99f0a0dbd5380c25aa9409fb763073f506ac050d Jun 02 01:32:19.078088 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.078054 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-k5t7m"] Jun 02 01:32:19.082945 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.082922 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-k5t7m" Jun 02 01:32:19.085278 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.085255 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-z4wzl\"" Jun 02 01:32:19.088021 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.087996 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-k5t7m"] Jun 02 01:32:19.164559 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.164523 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" event={"ID":"4c710341-1d2f-44c4-90f4-c86b3e3c2ed1","Type":"ContainerStarted","Data":"9682e3aa475770650238c73e99f0a0dbd5380c25aa9409fb763073f506ac050d"} Jun 02 01:32:19.164790 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.164760 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p24nq\" (UniqueName: \"kubernetes.io/projected/f4b0ffbb-6051-4e5f-b6da-33e58806855e-kube-api-access-p24nq\") pod \"authorino-7498df8756-k5t7m\" (UID: \"f4b0ffbb-6051-4e5f-b6da-33e58806855e\") " pod="kuadrant-system/authorino-7498df8756-k5t7m" Jun 02 01:32:19.266251 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.266154 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nq\" (UniqueName: \"kubernetes.io/projected/f4b0ffbb-6051-4e5f-b6da-33e58806855e-kube-api-access-p24nq\") pod \"authorino-7498df8756-k5t7m\" (UID: \"f4b0ffbb-6051-4e5f-b6da-33e58806855e\") " pod="kuadrant-system/authorino-7498df8756-k5t7m" Jun 02 01:32:19.273982 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.273955 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p24nq\" (UniqueName: \"kubernetes.io/projected/f4b0ffbb-6051-4e5f-b6da-33e58806855e-kube-api-access-p24nq\") pod \"authorino-7498df8756-k5t7m\" (UID: \"f4b0ffbb-6051-4e5f-b6da-33e58806855e\") " pod="kuadrant-system/authorino-7498df8756-k5t7m" Jun 02 01:32:19.393491 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.393456 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-k5t7m" Jun 02 01:32:19.565137 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:19.565036 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-k5t7m"] Jun 02 01:32:19.568918 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:32:19.568873 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4b0ffbb_6051_4e5f_b6da_33e58806855e.slice/crio-7ad655409366a273d7e94f79b7efde58902892830d6fe64fb5abe226547c3ecd WatchSource:0}: Error finding container 7ad655409366a273d7e94f79b7efde58902892830d6fe64fb5abe226547c3ecd: Status 404 returned error can't find the container with id 7ad655409366a273d7e94f79b7efde58902892830d6fe64fb5abe226547c3ecd Jun 02 01:32:20.171600 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:20.171555 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-k5t7m" event={"ID":"f4b0ffbb-6051-4e5f-b6da-33e58806855e","Type":"ContainerStarted","Data":"7ad655409366a273d7e94f79b7efde58902892830d6fe64fb5abe226547c3ecd"} Jun 02 01:32:24.187832 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:24.187798 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" event={"ID":"4c710341-1d2f-44c4-90f4-c86b3e3c2ed1","Type":"ContainerStarted","Data":"735e7100de92c5c04690ef213266ff945fb42366fdee38ccc2ad6ee20ffb4edb"} Jun 02 01:32:24.188374 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:24.187875 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:24.189019 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:24.188994 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-k5t7m" event={"ID":"f4b0ffbb-6051-4e5f-b6da-33e58806855e","Type":"ContainerStarted","Data":"ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d"} Jun 02 01:32:24.204367 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:24.204321 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" podStartSLOduration=1.881413448 podStartE2EDuration="6.204306578s" podCreationTimestamp="2026-06-02 01:32:18 +0000 UTC" firstStartedPulling="2026-06-02 01:32:19.064638216 +0000 UTC m=+567.269098490" lastFinishedPulling="2026-06-02 01:32:23.387531352 +0000 UTC m=+571.591991620" observedRunningTime="2026-06-02 01:32:24.202800707 +0000 UTC m=+572.407260990" watchObservedRunningTime="2026-06-02 01:32:24.204306578 +0000 UTC m=+572.408766860" Jun 02 01:32:35.193438 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:35.193399 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-9sm5p" Jun 02 01:32:35.209404 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:32:35.209356 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-k5t7m" podStartSLOduration=12.392847664 podStartE2EDuration="16.209342244s" podCreationTimestamp="2026-06-02 01:32:19 +0000 UTC" firstStartedPulling="2026-06-02 01:32:19.571092168 +0000 UTC m=+567.775552428" lastFinishedPulling="2026-06-02 01:32:23.387586741 +0000 UTC m=+571.592047008" observedRunningTime="2026-06-02 01:32:24.215405216 +0000 UTC m=+572.419865497" watchObservedRunningTime="2026-06-02 01:32:35.209342244 +0000 UTC m=+583.413802525" Jun 02 01:33:43.534794 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:43.534750 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-k5t7m"] Jun 02 01:33:43.535295 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:43.535000 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-k5t7m" podUID="f4b0ffbb-6051-4e5f-b6da-33e58806855e" containerName="authorino" containerID="cri-o://ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d" gracePeriod=30 Jun 02 01:33:43.788119 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:43.788054 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-k5t7m" Jun 02 01:33:43.839880 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:43.839845 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p24nq\" (UniqueName: \"kubernetes.io/projected/f4b0ffbb-6051-4e5f-b6da-33e58806855e-kube-api-access-p24nq\") pod \"f4b0ffbb-6051-4e5f-b6da-33e58806855e\" (UID: \"f4b0ffbb-6051-4e5f-b6da-33e58806855e\") " Jun 02 01:33:43.842076 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:43.842046 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b0ffbb-6051-4e5f-b6da-33e58806855e-kube-api-access-p24nq" (OuterVolumeSpecName: "kube-api-access-p24nq") pod "f4b0ffbb-6051-4e5f-b6da-33e58806855e" (UID: "f4b0ffbb-6051-4e5f-b6da-33e58806855e"). InnerVolumeSpecName "kube-api-access-p24nq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 01:33:43.941196 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:43.941152 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p24nq\" (UniqueName: \"kubernetes.io/projected/f4b0ffbb-6051-4e5f-b6da-33e58806855e-kube-api-access-p24nq\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:33:44.452547 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.452512 2564 generic.go:358] "Generic (PLEG): container finished" podID="f4b0ffbb-6051-4e5f-b6da-33e58806855e" containerID="ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d" exitCode=0 Jun 02 01:33:44.452726 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.452566 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-k5t7m" Jun 02 01:33:44.452726 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.452596 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-k5t7m" event={"ID":"f4b0ffbb-6051-4e5f-b6da-33e58806855e","Type":"ContainerDied","Data":"ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d"} Jun 02 01:33:44.452726 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.452634 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-k5t7m" event={"ID":"f4b0ffbb-6051-4e5f-b6da-33e58806855e","Type":"ContainerDied","Data":"7ad655409366a273d7e94f79b7efde58902892830d6fe64fb5abe226547c3ecd"} Jun 02 01:33:44.452726 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.452666 2564 scope.go:117] "RemoveContainer" containerID="ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d" Jun 02 01:33:44.460583 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.460563 2564 scope.go:117] "RemoveContainer" containerID="ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d" Jun 02 01:33:44.460867 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:33:44.460845 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d\": container with ID starting with ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d not found: ID does not exist" containerID="ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d" Jun 02 01:33:44.460948 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.460880 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d"} err="failed to get container status \"ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d\": rpc error: code = NotFound desc = could not find container \"ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d\": container with ID starting with ed1d7cad3c820c701ba7aef4fe775d174a66e55170b7bff840697f60512f1f5d not found: ID does not exist" Jun 02 01:33:44.466791 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.466763 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-k5t7m"] Jun 02 01:33:44.470146 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:44.470125 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-k5t7m"] Jun 02 01:33:46.378648 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:46.378607 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4b0ffbb-6051-4e5f-b6da-33e58806855e" path="/var/lib/kubelet/pods/f4b0ffbb-6051-4e5f-b6da-33e58806855e/volumes" Jun 02 01:33:46.996574 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:46.996540 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56979d96b8-85fhz"] Jun 02 01:33:46.996912 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:46.996899 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4b0ffbb-6051-4e5f-b6da-33e58806855e" containerName="authorino" Jun 02 01:33:46.996962 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:46.996913 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b0ffbb-6051-4e5f-b6da-33e58806855e" containerName="authorino" Jun 02 01:33:46.997007 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:46.996974 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="f4b0ffbb-6051-4e5f-b6da-33e58806855e" containerName="authorino" Jun 02 01:33:46.999428 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:46.999409 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:33:47.002164 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.002105 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 01:33:47.002448 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.002432 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qh8xs\"" Jun 02 01:33:47.009123 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.009091 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56979d96b8-85fhz"] Jun 02 01:33:47.071101 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.071052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtjtz\" (UniqueName: \"kubernetes.io/projected/cded6acd-6089-4c47-b74c-cfba23f26aad-kube-api-access-qtjtz\") pod \"maas-controller-56979d96b8-85fhz\" (UID: \"cded6acd-6089-4c47-b74c-cfba23f26aad\") " pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:33:47.172299 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.172249 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qtjtz\" (UniqueName: \"kubernetes.io/projected/cded6acd-6089-4c47-b74c-cfba23f26aad-kube-api-access-qtjtz\") pod \"maas-controller-56979d96b8-85fhz\" (UID: \"cded6acd-6089-4c47-b74c-cfba23f26aad\") " pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:33:47.193618 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.193582 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtjtz\" (UniqueName: \"kubernetes.io/projected/cded6acd-6089-4c47-b74c-cfba23f26aad-kube-api-access-qtjtz\") pod \"maas-controller-56979d96b8-85fhz\" (UID: \"cded6acd-6089-4c47-b74c-cfba23f26aad\") " pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:33:47.224942 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.224908 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f6cdb8b57-44wm7"] Jun 02 01:33:47.227498 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.227473 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" Jun 02 01:33:47.238496 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.238464 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f6cdb8b57-44wm7"] Jun 02 01:33:47.273718 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.273360 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjq5t\" (UniqueName: \"kubernetes.io/projected/d6aba4b2-0c53-4fb3-813f-a68ecf7657d1-kube-api-access-tjq5t\") pod \"maas-controller-6f6cdb8b57-44wm7\" (UID: \"d6aba4b2-0c53-4fb3-813f-a68ecf7657d1\") " pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" Jun 02 01:33:47.311138 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.311100 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:33:47.374482 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.374449 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tjq5t\" (UniqueName: \"kubernetes.io/projected/d6aba4b2-0c53-4fb3-813f-a68ecf7657d1-kube-api-access-tjq5t\") pod \"maas-controller-6f6cdb8b57-44wm7\" (UID: \"d6aba4b2-0c53-4fb3-813f-a68ecf7657d1\") " pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" Jun 02 01:33:47.383943 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.383865 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjq5t\" (UniqueName: \"kubernetes.io/projected/d6aba4b2-0c53-4fb3-813f-a68ecf7657d1-kube-api-access-tjq5t\") pod \"maas-controller-6f6cdb8b57-44wm7\" (UID: \"d6aba4b2-0c53-4fb3-813f-a68ecf7657d1\") " pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" Jun 02 01:33:47.451445 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.451340 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56979d96b8-85fhz"] Jun 02 01:33:47.453738 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:33:47.453706 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcded6acd_6089_4c47_b74c_cfba23f26aad.slice/crio-d68b0ea679911e435e7359e724813cc12d1ea094756c2129df54689a39dd2512 WatchSource:0}: Error finding container d68b0ea679911e435e7359e724813cc12d1ea094756c2129df54689a39dd2512: Status 404 returned error can't find the container with id d68b0ea679911e435e7359e724813cc12d1ea094756c2129df54689a39dd2512 Jun 02 01:33:47.464307 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.464273 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56979d96b8-85fhz" event={"ID":"cded6acd-6089-4c47-b74c-cfba23f26aad","Type":"ContainerStarted","Data":"d68b0ea679911e435e7359e724813cc12d1ea094756c2129df54689a39dd2512"} Jun 02 01:33:47.541100 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.540984 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" Jun 02 01:33:47.671107 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:47.671070 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f6cdb8b57-44wm7"] Jun 02 01:33:47.673921 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:33:47.673882 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6aba4b2_0c53_4fb3_813f_a68ecf7657d1.slice/crio-5c0f4dac98cbadeabe05b24b899d594cb9e7c0e51dea16ee843a2c86c54a7c1e WatchSource:0}: Error finding container 5c0f4dac98cbadeabe05b24b899d594cb9e7c0e51dea16ee843a2c86c54a7c1e: Status 404 returned error can't find the container with id 5c0f4dac98cbadeabe05b24b899d594cb9e7c0e51dea16ee843a2c86c54a7c1e Jun 02 01:33:48.471908 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:48.471812 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" event={"ID":"d6aba4b2-0c53-4fb3-813f-a68ecf7657d1","Type":"ContainerStarted","Data":"5c0f4dac98cbadeabe05b24b899d594cb9e7c0e51dea16ee843a2c86c54a7c1e"} Jun 02 01:33:50.480441 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:50.480341 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" event={"ID":"d6aba4b2-0c53-4fb3-813f-a68ecf7657d1","Type":"ContainerStarted","Data":"2a7f2395920eb185defa788b9d4b19819e9fbcfac5285decfdab19a0009cef8f"} Jun 02 01:33:50.480905 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:50.480471 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" Jun 02 01:33:50.481962 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:50.481933 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56979d96b8-85fhz" event={"ID":"cded6acd-6089-4c47-b74c-cfba23f26aad","Type":"ContainerStarted","Data":"878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc"} Jun 02 01:33:50.482073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:50.482054 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:33:50.501683 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:50.501590 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" podStartSLOduration=0.992956116 podStartE2EDuration="3.50157588s" podCreationTimestamp="2026-06-02 01:33:47 +0000 UTC" firstStartedPulling="2026-06-02 01:33:47.675310553 +0000 UTC m=+655.879770812" lastFinishedPulling="2026-06-02 01:33:50.18393031 +0000 UTC m=+658.388390576" observedRunningTime="2026-06-02 01:33:50.499036171 +0000 UTC m=+658.703496453" watchObservedRunningTime="2026-06-02 01:33:50.50157588 +0000 UTC m=+658.706036212" Jun 02 01:33:50.519463 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:33:50.519405 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-56979d96b8-85fhz" podStartSLOduration=1.794694077 podStartE2EDuration="4.519388049s" podCreationTimestamp="2026-06-02 01:33:46 +0000 UTC" firstStartedPulling="2026-06-02 01:33:47.455119873 +0000 UTC m=+655.659580133" lastFinishedPulling="2026-06-02 01:33:50.179813837 +0000 UTC m=+658.384274105" observedRunningTime="2026-06-02 01:33:50.517307241 +0000 UTC m=+658.721767526" watchObservedRunningTime="2026-06-02 01:33:50.519388049 +0000 UTC m=+658.723848372" Jun 02 01:34:01.490554 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:01.490522 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f6cdb8b57-44wm7" Jun 02 01:34:01.491206 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:01.490580 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:34:01.545007 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:01.544969 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56979d96b8-85fhz"] Jun 02 01:34:01.545227 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:01.545200 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-56979d96b8-85fhz" podUID="cded6acd-6089-4c47-b74c-cfba23f26aad" containerName="manager" containerID="cri-o://878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc" gracePeriod=30 Jun 02 01:34:01.788460 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:01.788434 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:34:01.912319 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:01.912278 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtjtz\" (UniqueName: \"kubernetes.io/projected/cded6acd-6089-4c47-b74c-cfba23f26aad-kube-api-access-qtjtz\") pod \"cded6acd-6089-4c47-b74c-cfba23f26aad\" (UID: \"cded6acd-6089-4c47-b74c-cfba23f26aad\") " Jun 02 01:34:01.914498 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:01.914466 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cded6acd-6089-4c47-b74c-cfba23f26aad-kube-api-access-qtjtz" (OuterVolumeSpecName: "kube-api-access-qtjtz") pod "cded6acd-6089-4c47-b74c-cfba23f26aad" (UID: "cded6acd-6089-4c47-b74c-cfba23f26aad"). InnerVolumeSpecName "kube-api-access-qtjtz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 01:34:02.013859 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.013817 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtjtz\" (UniqueName: \"kubernetes.io/projected/cded6acd-6089-4c47-b74c-cfba23f26aad-kube-api-access-qtjtz\") on node \"ip-10-0-133-218.ec2.internal\" DevicePath \"\"" Jun 02 01:34:02.525091 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.525057 2564 generic.go:358] "Generic (PLEG): container finished" podID="cded6acd-6089-4c47-b74c-cfba23f26aad" containerID="878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc" exitCode=0 Jun 02 01:34:02.525568 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.525125 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56979d96b8-85fhz" event={"ID":"cded6acd-6089-4c47-b74c-cfba23f26aad","Type":"ContainerDied","Data":"878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc"} Jun 02 01:34:02.525568 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.525135 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56979d96b8-85fhz" Jun 02 01:34:02.525568 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.525154 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56979d96b8-85fhz" event={"ID":"cded6acd-6089-4c47-b74c-cfba23f26aad","Type":"ContainerDied","Data":"d68b0ea679911e435e7359e724813cc12d1ea094756c2129df54689a39dd2512"} Jun 02 01:34:02.525568 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.525169 2564 scope.go:117] "RemoveContainer" containerID="878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc" Jun 02 01:34:02.533394 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.533376 2564 scope.go:117] "RemoveContainer" containerID="878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc" Jun 02 01:34:02.533677 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:34:02.533647 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc\": container with ID starting with 878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc not found: ID does not exist" containerID="878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc" Jun 02 01:34:02.533726 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.533687 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc"} err="failed to get container status \"878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc\": rpc error: code = NotFound desc = could not find container \"878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc\": container with ID starting with 878d94141e9d28a0cbda18646603ebf89fad510eb4f2bc95912844b64eed68dc not found: ID does not exist" Jun 02 01:34:02.541282 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.541253 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56979d96b8-85fhz"] Jun 02 01:34:02.544203 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:02.544176 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56979d96b8-85fhz"] Jun 02 01:34:04.378382 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:04.378347 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cded6acd-6089-4c47-b74c-cfba23f26aad" path="/var/lib/kubelet/pods/cded6acd-6089-4c47-b74c-cfba23f26aad/volumes" Jun 02 01:34:20.843577 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.843540 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-d8878fcc5-dzdj7"] Jun 02 01:34:20.844071 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.843899 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cded6acd-6089-4c47-b74c-cfba23f26aad" containerName="manager" Jun 02 01:34:20.844071 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.843911 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cded6acd-6089-4c47-b74c-cfba23f26aad" containerName="manager" Jun 02 01:34:20.844071 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.843992 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="cded6acd-6089-4c47-b74c-cfba23f26aad" containerName="manager" Jun 02 01:34:20.848324 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.848299 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:20.850906 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.850878 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 02 01:34:20.851052 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.850890 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-xcvv7\"" Jun 02 01:34:20.858430 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.858404 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-d8878fcc5-dzdj7"] Jun 02 01:34:20.879994 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.879954 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49b5b854-fc3d-4288-aab5-567b8e42a689-maas-api-tls\") pod \"maas-api-d8878fcc5-dzdj7\" (UID: \"49b5b854-fc3d-4288-aab5-567b8e42a689\") " pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:20.880175 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.880035 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnr7f\" (UniqueName: \"kubernetes.io/projected/49b5b854-fc3d-4288-aab5-567b8e42a689-kube-api-access-hnr7f\") pod \"maas-api-d8878fcc5-dzdj7\" (UID: \"49b5b854-fc3d-4288-aab5-567b8e42a689\") " pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:20.981280 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.981242 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49b5b854-fc3d-4288-aab5-567b8e42a689-maas-api-tls\") pod \"maas-api-d8878fcc5-dzdj7\" (UID: \"49b5b854-fc3d-4288-aab5-567b8e42a689\") " pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:20.981461 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.981306 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hnr7f\" (UniqueName: \"kubernetes.io/projected/49b5b854-fc3d-4288-aab5-567b8e42a689-kube-api-access-hnr7f\") pod \"maas-api-d8878fcc5-dzdj7\" (UID: \"49b5b854-fc3d-4288-aab5-567b8e42a689\") " pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:20.981461 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:34:20.981393 2564 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 02 01:34:20.981542 ip-10-0-133-218 kubenswrapper[2564]: E0602 01:34:20.981471 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49b5b854-fc3d-4288-aab5-567b8e42a689-maas-api-tls podName:49b5b854-fc3d-4288-aab5-567b8e42a689 nodeName:}" failed. No retries permitted until 2026-06-02 01:34:21.481454269 +0000 UTC m=+689.685914529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49b5b854-fc3d-4288-aab5-567b8e42a689-maas-api-tls") pod "maas-api-d8878fcc5-dzdj7" (UID: "49b5b854-fc3d-4288-aab5-567b8e42a689") : secret "maas-api-serving-cert" not found Jun 02 01:34:20.991482 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:20.991448 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnr7f\" (UniqueName: \"kubernetes.io/projected/49b5b854-fc3d-4288-aab5-567b8e42a689-kube-api-access-hnr7f\") pod \"maas-api-d8878fcc5-dzdj7\" (UID: \"49b5b854-fc3d-4288-aab5-567b8e42a689\") " pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:21.446025 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:21.445980 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:34:21.486220 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:21.486177 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49b5b854-fc3d-4288-aab5-567b8e42a689-maas-api-tls\") pod \"maas-api-d8878fcc5-dzdj7\" (UID: \"49b5b854-fc3d-4288-aab5-567b8e42a689\") " pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:21.488934 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:21.488895 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49b5b854-fc3d-4288-aab5-567b8e42a689-maas-api-tls\") pod \"maas-api-d8878fcc5-dzdj7\" (UID: \"49b5b854-fc3d-4288-aab5-567b8e42a689\") " pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:21.759957 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:21.759840 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:21.897354 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:21.897326 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-d8878fcc5-dzdj7"] Jun 02 01:34:21.900475 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:34:21.900438 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b5b854_fc3d_4288_aab5_567b8e42a689.slice/crio-af510adcceff572f3bbef593d974381284e44533ac85d2d6fef0ae666c012afe WatchSource:0}: Error finding container af510adcceff572f3bbef593d974381284e44533ac85d2d6fef0ae666c012afe: Status 404 returned error can't find the container with id af510adcceff572f3bbef593d974381284e44533ac85d2d6fef0ae666c012afe Jun 02 01:34:21.902118 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:21.902096 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 01:34:22.594480 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:22.594433 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-d8878fcc5-dzdj7" event={"ID":"49b5b854-fc3d-4288-aab5-567b8e42a689","Type":"ContainerStarted","Data":"af510adcceff572f3bbef593d974381284e44533ac85d2d6fef0ae666c012afe"} Jun 02 01:34:24.604566 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:24.604529 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-d8878fcc5-dzdj7" event={"ID":"49b5b854-fc3d-4288-aab5-567b8e42a689","Type":"ContainerStarted","Data":"2c1cc120731cd7bf664763f8efabbae65701bff083b50786e0a6391d3af9409a"} Jun 02 01:34:24.604995 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:24.604696 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:34:24.621809 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:24.621751 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-d8878fcc5-dzdj7" podStartSLOduration=2.979138307 podStartE2EDuration="4.62173564s" podCreationTimestamp="2026-06-02 01:34:20 +0000 UTC" firstStartedPulling="2026-06-02 01:34:21.902229028 +0000 UTC m=+690.106689288" lastFinishedPulling="2026-06-02 01:34:23.544826358 +0000 UTC m=+691.749286621" observedRunningTime="2026-06-02 01:34:24.619500107 +0000 UTC m=+692.823960386" watchObservedRunningTime="2026-06-02 01:34:24.62173564 +0000 UTC m=+692.826195921" Jun 02 01:34:30.613936 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:34:30.613902 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-d8878fcc5-dzdj7" Jun 02 01:35:10.755827 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.755787 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c5ddf5db-d52zm"] Jun 02 01:35:10.760544 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.760522 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.763932 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.763903 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-z4wzl\"" Jun 02 01:35:10.764073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.763917 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 02 01:35:10.764073 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.763970 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 02 01:35:10.769088 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.769062 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c5ddf5db-d52zm"] Jun 02 01:35:10.826108 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.826063 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/35fe94e7-5314-449b-b72d-8747e96ae587-tls-cert\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.826280 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.826121 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/35fe94e7-5314-449b-b72d-8747e96ae587-oidc-ca\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.826280 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.826191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p87nq\" (UniqueName: \"kubernetes.io/projected/35fe94e7-5314-449b-b72d-8747e96ae587-kube-api-access-p87nq\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.926988 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.926952 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/35fe94e7-5314-449b-b72d-8747e96ae587-tls-cert\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.926988 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.927004 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/35fe94e7-5314-449b-b72d-8747e96ae587-oidc-ca\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.927237 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.927047 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p87nq\" (UniqueName: \"kubernetes.io/projected/35fe94e7-5314-449b-b72d-8747e96ae587-kube-api-access-p87nq\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.927774 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.927751 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/35fe94e7-5314-449b-b72d-8747e96ae587-oidc-ca\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.929484 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.929461 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/35fe94e7-5314-449b-b72d-8747e96ae587-tls-cert\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:10.934000 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:10.933974 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p87nq\" (UniqueName: \"kubernetes.io/projected/35fe94e7-5314-449b-b72d-8747e96ae587-kube-api-access-p87nq\") pod \"authorino-59c5ddf5db-d52zm\" (UID: \"35fe94e7-5314-449b-b72d-8747e96ae587\") " pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:11.071019 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:11.070929 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c5ddf5db-d52zm" Jun 02 01:35:11.198111 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:11.198073 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c5ddf5db-d52zm"] Jun 02 01:35:11.200798 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:35:11.200766 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35fe94e7_5314_449b_b72d_8747e96ae587.slice/crio-e8b422ca9cea75522f1c3c6b4e68f1b863d835084fb664ef327abab637f9e9fb WatchSource:0}: Error finding container e8b422ca9cea75522f1c3c6b4e68f1b863d835084fb664ef327abab637f9e9fb: Status 404 returned error can't find the container with id e8b422ca9cea75522f1c3c6b4e68f1b863d835084fb664ef327abab637f9e9fb Jun 02 01:35:11.764707 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:11.764673 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c5ddf5db-d52zm" event={"ID":"35fe94e7-5314-449b-b72d-8747e96ae587","Type":"ContainerStarted","Data":"432c08487851e83673ea6805390ab38e1986c5e5bdbef388c7825c5d689d7cb9"} Jun 02 01:35:11.764707 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:11.764711 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c5ddf5db-d52zm" event={"ID":"35fe94e7-5314-449b-b72d-8747e96ae587","Type":"ContainerStarted","Data":"e8b422ca9cea75522f1c3c6b4e68f1b863d835084fb664ef327abab637f9e9fb"} Jun 02 01:35:11.785469 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:11.785366 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-59c5ddf5db-d52zm" podStartSLOduration=1.470630716 podStartE2EDuration="1.785345955s" podCreationTimestamp="2026-06-02 01:35:10 +0000 UTC" firstStartedPulling="2026-06-02 01:35:11.202144356 +0000 UTC m=+739.406604628" lastFinishedPulling="2026-06-02 01:35:11.516859602 +0000 UTC m=+739.721319867" observedRunningTime="2026-06-02 01:35:11.783070521 +0000 UTC m=+739.987530793" watchObservedRunningTime="2026-06-02 01:35:11.785345955 +0000 UTC m=+739.989806215" Jun 02 01:35:22.293412 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:22.293370 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:35:26.799680 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:26.799627 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:35:33.307428 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.307394 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f"] Jun 02 01:35:33.311397 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.311375 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.313900 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.313868 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 01:35:33.314897 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.314878 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 02 01:35:33.315012 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.314923 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hpnsh\"" Jun 02 01:35:33.315012 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.314947 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 01:35:33.320095 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.320071 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f"] Jun 02 01:35:33.446903 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.446843 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.446903 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.446908 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktg2q\" (UniqueName: \"kubernetes.io/projected/3d46c025-cd85-4d35-9c82-497f3cc19219-kube-api-access-ktg2q\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.447157 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.446952 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.447157 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.447042 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.447157 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.447090 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.447157 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.447110 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d46c025-cd85-4d35-9c82-497f3cc19219-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548255 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548203 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548255 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548256 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ktg2q\" (UniqueName: \"kubernetes.io/projected/3d46c025-cd85-4d35-9c82-497f3cc19219-kube-api-access-ktg2q\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548529 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548276 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548529 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548323 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548529 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548359 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548529 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548385 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d46c025-cd85-4d35-9c82-497f3cc19219-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548839 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548808 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548969 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548857 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.548969 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.548884 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.550578 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.550561 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d46c025-cd85-4d35-9c82-497f3cc19219-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.550973 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.550954 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d46c025-cd85-4d35-9c82-497f3cc19219-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.556898 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.556869 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktg2q\" (UniqueName: \"kubernetes.io/projected/3d46c025-cd85-4d35-9c82-497f3cc19219-kube-api-access-ktg2q\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f\" (UID: \"3d46c025-cd85-4d35-9c82-497f3cc19219\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.624172 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.624136 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:33.757872 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.757829 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f"] Jun 02 01:35:33.760219 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:35:33.760184 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d46c025_cd85_4d35_9c82_497f3cc19219.slice/crio-5bfce8aad654d12d544a4f0b17844c162a7cbbc1b7287947675094638e0ec0f7 WatchSource:0}: Error finding container 5bfce8aad654d12d544a4f0b17844c162a7cbbc1b7287947675094638e0ec0f7: Status 404 returned error can't find the container with id 5bfce8aad654d12d544a4f0b17844c162a7cbbc1b7287947675094638e0ec0f7 Jun 02 01:35:33.838867 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:33.838832 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" event={"ID":"3d46c025-cd85-4d35-9c82-497f3cc19219","Type":"ContainerStarted","Data":"5bfce8aad654d12d544a4f0b17844c162a7cbbc1b7287947675094638e0ec0f7"} Jun 02 01:35:34.295501 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:34.295432 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:35:37.857183 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:37.857145 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" event={"ID":"3d46c025-cd85-4d35-9c82-497f3cc19219","Type":"ContainerStarted","Data":"d52ca5314209ebbd0d8736262bc0ff32b6a64e935cb2b80cae8259a8c7e23360"} Jun 02 01:35:38.861875 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:38.861839 2564 generic.go:358] "Generic (PLEG): container finished" podID="3d46c025-cd85-4d35-9c82-497f3cc19219" containerID="d52ca5314209ebbd0d8736262bc0ff32b6a64e935cb2b80cae8259a8c7e23360" exitCode=0 Jun 02 01:35:38.862234 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:38.861922 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" event={"ID":"3d46c025-cd85-4d35-9c82-497f3cc19219","Type":"ContainerDied","Data":"d52ca5314209ebbd0d8736262bc0ff32b6a64e935cb2b80cae8259a8c7e23360"} Jun 02 01:35:40.872025 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:40.871984 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" event={"ID":"3d46c025-cd85-4d35-9c82-497f3cc19219","Type":"ContainerStarted","Data":"364958e3fd56a5b3735b7c8d9f38b041f27cf992ff62fc43ccdb7de1a18c4dc7"} Jun 02 01:35:40.890683 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:40.890615 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" podStartSLOduration=1.708740241 podStartE2EDuration="7.89060163s" podCreationTimestamp="2026-06-02 01:35:33 +0000 UTC" firstStartedPulling="2026-06-02 01:35:33.762116172 +0000 UTC m=+761.966576432" lastFinishedPulling="2026-06-02 01:35:39.943977561 +0000 UTC m=+768.148437821" observedRunningTime="2026-06-02 01:35:40.887994006 +0000 UTC m=+769.092454287" watchObservedRunningTime="2026-06-02 01:35:40.89060163 +0000 UTC m=+769.095061912" Jun 02 01:35:43.625256 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:43.625214 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:43.625256 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:43.625262 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:43.638314 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:43.638282 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:43.696134 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:43.696095 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:35:43.896025 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:43.895945 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f" Jun 02 01:35:50.601900 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:50.601858 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:35:58.895826 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.895789 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65"] Jun 02 01:35:58.912090 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.912051 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:58.913312 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.913286 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65"] Jun 02 01:35:58.914535 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.914510 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 02 01:35:58.987567 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.987525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:58.987567 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.987572 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a27e4f19-0494-4f6c-8485-7d15bad36ee5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:58.987832 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.987607 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:58.987832 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.987714 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:58.987832 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.987746 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:58.987832 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:58.987769 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcxfb\" (UniqueName: \"kubernetes.io/projected/a27e4f19-0494-4f6c-8485-7d15bad36ee5-kube-api-access-mcxfb\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088264 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088230 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088453 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088275 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088453 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088298 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mcxfb\" (UniqueName: \"kubernetes.io/projected/a27e4f19-0494-4f6c-8485-7d15bad36ee5-kube-api-access-mcxfb\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088453 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088332 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088453 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088350 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a27e4f19-0494-4f6c-8485-7d15bad36ee5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088453 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088376 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088735 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088706 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088735 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088719 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.088826 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.088801 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.090554 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.090529 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a27e4f19-0494-4f6c-8485-7d15bad36ee5-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.091002 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.090980 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a27e4f19-0494-4f6c-8485-7d15bad36ee5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.096013 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.095985 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcxfb\" (UniqueName: \"kubernetes.io/projected/a27e4f19-0494-4f6c-8485-7d15bad36ee5-kube-api-access-mcxfb\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65\" (UID: \"a27e4f19-0494-4f6c-8485-7d15bad36ee5\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.226903 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.226823 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" Jun 02 01:35:59.357779 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.357751 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65"] Jun 02 01:35:59.359692 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:35:59.359648 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27e4f19_0494_4f6c_8485_7d15bad36ee5.slice/crio-80c5b70e5ca7ec3d3216c16eef93ee91d80bdd9c8a5f405784ca4976835e4162 WatchSource:0}: Error finding container 80c5b70e5ca7ec3d3216c16eef93ee91d80bdd9c8a5f405784ca4976835e4162: Status 404 returned error can't find the container with id 80c5b70e5ca7ec3d3216c16eef93ee91d80bdd9c8a5f405784ca4976835e4162 Jun 02 01:35:59.941098 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.941058 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" event={"ID":"a27e4f19-0494-4f6c-8485-7d15bad36ee5","Type":"ContainerStarted","Data":"928a6d5aa37dfcbb67776e9d30a3f47f1be002c0fe6ad5efaab27d63f36f0d6c"} Jun 02 01:35:59.941098 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:35:59.941101 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65" event={"ID":"a27e4f19-0494-4f6c-8485-7d15bad36ee5","Type":"ContainerStarted","Data":"80c5b70e5ca7ec3d3216c16eef93ee91d80bdd9c8a5f405784ca4976835e4162"} Jun 02 01:36:01.302491 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:36:01.302445 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-9sm5p"] Jun 02 01:42:02.785111 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:02.785078 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65_a27e4f19-0494-4f6c-8485-7d15bad36ee5/storage-initializer/0.log" Jun 02 01:42:03.747074 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:03.747042 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c5ddf5db-d52zm_35fe94e7-5314-449b-b72d-8747e96ae587/authorino/0.log" Jun 02 01:42:08.788857 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:08.788823 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-d8878fcc5-dzdj7_49b5b854-fc3d-4288-aab5-567b8e42a689/maas-api/0.log" Jun 02 01:42:08.910295 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:08.910254 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f6cdb8b57-44wm7_d6aba4b2-0c53-4fb3-813f-a68ecf7657d1/manager/0.log" Jun 02 01:42:09.388863 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:09.388834 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6758f7c588-dxhts_830114f5-92cf-4d17-81fe-cf1e7efcf37c/copy-manifests/0.log" Jun 02 01:42:09.402859 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:09.402829 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6758f7c588-dxhts_830114f5-92cf-4d17-81fe-cf1e7efcf37c/manager/0.log" Jun 02 01:42:10.926631 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:10.926584 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c5ddf5db-d52zm_35fe94e7-5314-449b-b72d-8747e96ae587/authorino/0.log" Jun 02 01:42:11.261007 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:11.260927 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-bv42w_0f96b4b5-9080-4e65-8b35-220895896adc/kuadrant-console-plugin/0.log" Jun 02 01:42:11.601610 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:11.601531 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-9sm5p_4c710341-1d2f-44c4-90f4-c86b3e3c2ed1/limitador/0.log" Jun 02 01:42:12.407839 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:12.407752 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-865b6fd47-znw7r_84057c87-4bac-4372-8add-f7bd2ece8060/kube-auth-proxy/0.log" Jun 02 01:42:12.739315 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:12.739237 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c7cfdd785-nlm5p_16360c44-5de0-4680-99d0-0b9e9120ec63/router/0.log" Jun 02 01:42:13.328954 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:13.328925 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f_3d46c025-cd85-4d35-9c82-497f3cc19219/storage-initializer/0.log" Jun 02 01:42:13.335808 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:13.335787 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-bx92f_3d46c025-cd85-4d35-9c82-497f3cc19219/main/0.log" Jun 02 01:42:13.685499 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:13.685466 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-k9w65_a27e4f19-0494-4f6c-8485-7d15bad36ee5/storage-initializer/0.log" Jun 02 01:42:20.851533 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:20.851496 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4vq4w_ab2a9f80-37fa-4aa6-bc80-1914f1967e78/global-pull-secret-syncer/0.log" Jun 02 01:42:21.045822 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:21.045792 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xkf6n_52b5b39e-4c8a-48ed-beae-191f194d012b/konnectivity-agent/0.log" Jun 02 01:42:21.089193 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:21.089161 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-218.ec2.internal_645034bd148c3fd0460f6af9403bc9a0/haproxy/0.log" Jun 02 01:42:25.372463 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:25.372428 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c5ddf5db-d52zm_35fe94e7-5314-449b-b72d-8747e96ae587/authorino/0.log" Jun 02 01:42:25.448008 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:25.447980 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-bv42w_0f96b4b5-9080-4e65-8b35-220895896adc/kuadrant-console-plugin/0.log" Jun 02 01:42:25.542007 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:25.541929 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-9sm5p_4c710341-1d2f-44c4-90f4-c86b3e3c2ed1/limitador/0.log" Jun 02 01:42:26.998680 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:26.998634 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-69b6cbdb88-q5v5z_3e5e657c-d1d2-4183-aae9-86ede8ed3897/cluster-monitoring-operator/0.log" Jun 02 01:42:27.023204 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.023181 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-8gnlt_61b4e7c0-a11b-4424-9664-a14c00c94b4e/kube-state-metrics/0.log" Jun 02 01:42:27.039895 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.039866 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-8gnlt_61b4e7c0-a11b-4424-9664-a14c00c94b4e/kube-rbac-proxy-main/0.log" Jun 02 01:42:27.061060 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.061032 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-8gnlt_61b4e7c0-a11b-4424-9664-a14c00c94b4e/kube-rbac-proxy-self/0.log" Jun 02 01:42:27.091546 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.091521 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-675b845476-klc5n_762b97d5-b17b-432b-be89-715419a3d70f/metrics-server/0.log" Jun 02 01:42:27.129030 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.129006 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-wnwvt_d3ad6a3c-8841-4dd1-9309-91975b7ff7c8/monitoring-plugin/0.log" Jun 02 01:42:27.370213 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.370179 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rrk44_51bbd087-24d1-475d-ba30-bedf031a883c/node-exporter/0.log" Jun 02 01:42:27.396434 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.396411 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rrk44_51bbd087-24d1-475d-ba30-bedf031a883c/kube-rbac-proxy/0.log" Jun 02 01:42:27.428099 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.428072 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rrk44_51bbd087-24d1-475d-ba30-bedf031a883c/init-textfile/0.log" Jun 02 01:42:27.536173 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.536100 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/prometheus/0.log" Jun 02 01:42:27.554706 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.554677 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/config-reloader/0.log" Jun 02 01:42:27.574129 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.574104 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/thanos-sidecar/0.log" Jun 02 01:42:27.596512 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.596486 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/kube-rbac-proxy-web/0.log" Jun 02 01:42:27.616777 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.616756 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/kube-rbac-proxy/0.log" Jun 02 01:42:27.637649 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.637622 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/kube-rbac-proxy-thanos/0.log" Jun 02 01:42:27.659244 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.659218 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d5c31886-d58e-4c6f-8efa-9b32271e7a75/init-config-reloader/0.log" Jun 02 01:42:27.849373 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.849293 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/thanos-query/0.log" Jun 02 01:42:27.869491 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.869459 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/kube-rbac-proxy-web/0.log" Jun 02 01:42:27.888432 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.888404 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/kube-rbac-proxy/0.log" Jun 02 01:42:27.907563 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.907533 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/prom-label-proxy/0.log" Jun 02 01:42:27.928461 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.928433 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/kube-rbac-proxy-rules/0.log" Jun 02 01:42:27.947841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:27.947816 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78987677b8-nptt4_91ac6667-5311-4e0a-93a1-dad9f25b6713/kube-rbac-proxy-metrics/0.log" Jun 02 01:42:29.058377 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:29.058348 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-qm5j8_fb11ade2-f6f5-4fea-bf71-8193a5071c03/networking-console-plugin/0.log" Jun 02 01:42:30.133911 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.133877 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-zsj96_097f5758-d818-44c7-bd95-39a8e63b75e0/download-server/0.log" Jun 02 01:42:30.474739 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.474668 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz"] Jun 02 01:42:30.478292 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.478275 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.480605 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.480581 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-524c7\"/\"kube-root-ca.crt\"" Jun 02 01:42:30.480605 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.480602 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-524c7\"/\"openshift-service-ca.crt\"" Jun 02 01:42:30.480806 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.480682 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-524c7\"/\"default-dockercfg-45n7s\"" Jun 02 01:42:30.484546 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.484524 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz"] Jun 02 01:42:30.581455 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.581407 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rl2z\" (UniqueName: \"kubernetes.io/projected/3bfd79e7-af23-4133-a5db-c83fc3cab885-kube-api-access-4rl2z\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.581455 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.581455 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-sys\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.581765 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.581560 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-proc\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.581765 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.581611 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-lib-modules\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.581765 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.581647 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-podres\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.682768 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.682724 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-sys\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.682941 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.682852 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-sys\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.682941 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.682863 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-proc\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.682941 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.682917 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-proc\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.682941 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.682916 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-lib-modules\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.683072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.682952 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-podres\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.683072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.683003 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-lib-modules\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.683072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.683015 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4rl2z\" (UniqueName: \"kubernetes.io/projected/3bfd79e7-af23-4133-a5db-c83fc3cab885-kube-api-access-4rl2z\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.683072 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.683057 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3bfd79e7-af23-4133-a5db-c83fc3cab885-podres\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.690395 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.690360 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rl2z\" (UniqueName: \"kubernetes.io/projected/3bfd79e7-af23-4133-a5db-c83fc3cab885-kube-api-access-4rl2z\") pod \"perf-node-gather-daemonset-9nrvz\" (UID: \"3bfd79e7-af23-4133-a5db-c83fc3cab885\") " pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.790107 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.789988 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:30.929418 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.924112 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz"] Jun 02 01:42:30.931059 ip-10-0-133-218 kubenswrapper[2564]: W0602 01:42:30.931019 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3bfd79e7_af23_4133_a5db_c83fc3cab885.slice/crio-315e64bc0b692242825cfdb75043c92fe77bba72c25d2b33de9309c25e614d0f WatchSource:0}: Error finding container 315e64bc0b692242825cfdb75043c92fe77bba72c25d2b33de9309c25e614d0f: Status 404 returned error can't find the container with id 315e64bc0b692242825cfdb75043c92fe77bba72c25d2b33de9309c25e614d0f Jun 02 01:42:30.933025 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:30.933006 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 01:42:31.309759 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:31.309717 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" event={"ID":"3bfd79e7-af23-4133-a5db-c83fc3cab885","Type":"ContainerStarted","Data":"e4435c49cb1b72ed8e03adfc7a09d07fe51188b5356bd7e44564c15271582fc8"} Jun 02 01:42:31.309759 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:31.309758 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" event={"ID":"3bfd79e7-af23-4133-a5db-c83fc3cab885","Type":"ContainerStarted","Data":"315e64bc0b692242825cfdb75043c92fe77bba72c25d2b33de9309c25e614d0f"} Jun 02 01:42:31.310301 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:31.309816 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:31.325414 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:31.325358 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" podStartSLOduration=1.325343122 podStartE2EDuration="1.325343122s" podCreationTimestamp="2026-06-02 01:42:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 01:42:31.323696311 +0000 UTC m=+1179.528156592" watchObservedRunningTime="2026-06-02 01:42:31.325343122 +0000 UTC m=+1179.529803448" Jun 02 01:42:31.575320 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:31.575236 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tfpsh_0a88a237-04e2-41a9-a1c5-dedfb5bc6df0/dns/0.log" Jun 02 01:42:31.595416 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:31.595377 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tfpsh_0a88a237-04e2-41a9-a1c5-dedfb5bc6df0/kube-rbac-proxy/0.log" Jun 02 01:42:31.715140 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:31.715101 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wn6w8_234e4e76-252c-491b-b14f-6b9a7e2c36b9/dns-node-resolver/0.log" Jun 02 01:42:32.131841 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:32.131814 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7f8755f774-lbkfj_0362e1c2-f61f-4871-a49a-348ad82648e6/registry/0.log" Jun 02 01:42:32.172140 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:32.172112 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8lt9m_a415cede-b0c5-4300-8f99-22397439fe3f/node-ca/0.log" Jun 02 01:42:33.096820 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:33.096787 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-865b6fd47-znw7r_84057c87-4bac-4372-8add-f7bd2ece8060/kube-auth-proxy/0.log" Jun 02 01:42:33.172177 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:33.172136 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c7cfdd785-nlm5p_16360c44-5de0-4680-99d0-0b9e9120ec63/router/0.log" Jun 02 01:42:33.653275 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:33.653244 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gtr8q_99aff132-a1e2-4586-80cb-cee5ad8238bc/serve-healthcheck-canary/0.log" Jun 02 01:42:34.300100 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:34.300071 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zf9vl_85025c29-0ce5-42b2-a10f-04fac970bb3f/kube-rbac-proxy/0.log" Jun 02 01:42:34.318484 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:34.318446 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zf9vl_85025c29-0ce5-42b2-a10f-04fac970bb3f/exporter/0.log" Jun 02 01:42:34.340175 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:34.340142 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zf9vl_85025c29-0ce5-42b2-a10f-04fac970bb3f/extractor/0.log" Jun 02 01:42:36.416453 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:36.416429 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-d8878fcc5-dzdj7_49b5b854-fc3d-4288-aab5-567b8e42a689/maas-api/0.log" Jun 02 01:42:36.441484 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:36.441452 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f6cdb8b57-44wm7_d6aba4b2-0c53-4fb3-813f-a68ecf7657d1/manager/0.log" Jun 02 01:42:36.609555 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:36.609514 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6758f7c588-dxhts_830114f5-92cf-4d17-81fe-cf1e7efcf37c/manager/0.log" Jun 02 01:42:36.625586 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:36.625560 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6758f7c588-dxhts_830114f5-92cf-4d17-81fe-cf1e7efcf37c/copy-manifests/0.log" Jun 02 01:42:37.323231 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:37.323197 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-524c7/perf-node-gather-daemonset-9nrvz" Jun 02 01:42:37.926670 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:37.926630 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6df7964bff-gqrnr_29b5c3aa-0f07-4ccf-b9e8-c855e71b65d7/manager/0.log" Jun 02 01:42:44.022395 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.022361 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-672sz_401e1384-3579-4929-86af-eb642d871287/kube-multus/0.log" Jun 02 01:42:44.083988 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.083961 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tmcl_e85ff609-4afc-446a-83cd-77f70781dd24/kube-multus-additional-cni-plugins/0.log" Jun 02 01:42:44.104153 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.104126 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tmcl_e85ff609-4afc-446a-83cd-77f70781dd24/egress-router-binary-copy/0.log" Jun 02 01:42:44.123750 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.123717 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tmcl_e85ff609-4afc-446a-83cd-77f70781dd24/cni-plugins/0.log" Jun 02 01:42:44.144272 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.144243 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tmcl_e85ff609-4afc-446a-83cd-77f70781dd24/bond-cni-plugin/0.log" Jun 02 01:42:44.164702 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.164677 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tmcl_e85ff609-4afc-446a-83cd-77f70781dd24/routeoverride-cni/0.log" Jun 02 01:42:44.183662 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.183615 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tmcl_e85ff609-4afc-446a-83cd-77f70781dd24/whereabouts-cni-bincopy/0.log" Jun 02 01:42:44.202161 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.202129 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tmcl_e85ff609-4afc-446a-83cd-77f70781dd24/whereabouts-cni/0.log" Jun 02 01:42:44.617596 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.617568 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zvkvb_cf148b2d-3a22-4a94-98f3-5dcc28eebd87/network-metrics-daemon/0.log" Jun 02 01:42:44.634931 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:44.634903 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zvkvb_cf148b2d-3a22-4a94-98f3-5dcc28eebd87/kube-rbac-proxy/0.log" Jun 02 01:42:45.633326 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:45.633279 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qsmk7_962fbf55-761d-4399-9cd3-6d3cfb488044/ovn-controller/0.log" Jun 02 01:42:45.658913 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:45.658886 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qsmk7_962fbf55-761d-4399-9cd3-6d3cfb488044/ovn-acl-logging/0.log" Jun 02 01:42:45.675792 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:45.675766 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qsmk7_962fbf55-761d-4399-9cd3-6d3cfb488044/kube-rbac-proxy-node/0.log" Jun 02 01:42:45.697024 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:45.696994 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qsmk7_962fbf55-761d-4399-9cd3-6d3cfb488044/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 01:42:45.718804 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:45.718779 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qsmk7_962fbf55-761d-4399-9cd3-6d3cfb488044/northd/0.log" Jun 02 01:42:45.739037 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:45.739013 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qsmk7_962fbf55-761d-4399-9cd3-6d3cfb488044/nbdb/0.log" Jun 02 01:42:45.758490 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:45.758465 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qsmk7_962fbf55-761d-4399-9cd3-6d3cfb488044/sbdb/0.log" Jun 02 01:42:45.867714 ip-10-0-133-218 kubenswrapper[2564]: I0602 01:42:45.867679 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qsmk7_962fbf55-761d-4399-9cd3-6d3cfb488044/ovnkube-controller/0.log"